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Cartan-decomposition subgroups of SU(2,n) 



Alessandra lozzi and Dave Witte 



Abstract. We give explicit, practical conditions that determine whether or 
not a closed, connected subgroup H oi G = SU(2, n) has the property that 
there exists a compact subset C oi G with CHC = G. To do this, we fix a 
Cartan decomposition G = KA^K of G, and then carry out an approximate 
calculation of {KHK) n for each closed, connected subgroup H oi G . This 
generalizes the work of H. Oh and D. Witte for G ~ S0(2, n) . 



1 Introduction 

Definition 1.1. [14, Defn. 1.2] Let if be a closed subgroup of a connected, 
simple, linear, real Lie group G. We say that if is a Cartan-decomposition 
subgroup of G if 

• H is connected, and 

• there is a compact subset C of G , such that CHC = G. 

(Note that C is only assumed to be a subset of G; it need not be a subgroup.) 

Example 1.2. The Cartan decomposition G = KAK shows that the maximal 
split torus A is a Cartan-decomposition subgroup of G . 

It is known that G = KNK [9, Thm. 5.1], so the maximal unipotent 
subgroup is also a Cartan-decomposition subgroup. 

If M-rankG = (that is, if G is compact), then every (closed, connected) 
subgroup of G is a Cartan-decomposition subgroup. 

If M-rank G = 1 , then it not difficult to see that every (closed, connected) 
noncompact subgroup of G is a Cartan-decomposition subgroup (cf. [5, Lem. 3.2]). 

It is more difficult to characterize the Cartan-decomposition subgroups 
when M-rankG = 2, but H. Oh and D. Witte [14] studied two examples in detail. 
Namely, they described all the Cartan-decomposition subgroups of SL(3,M) and 
of S0(2,n), and they also explicitly described the closed, connected subgroups 
that are not Cartan-decomposition subgroups. Here, we obtain similar results for 
SU(2, n) . Unfortunately, the results are rather complicated to state. 

Notation 1.3. Let G = SU(2,n) and fix an Iwasawa decomposition G = 
KAN and a corresponding Cartan decomposition G = KA^K., where A^ is the 
(closed) positive Weyl chamber of A in which the roots occurring in the Lie algebra 
of N are positive. Thus, i^ is a maximal compact subgroup, A is the identity 
component of a maximal split torus, and iV is a maximal unipotent subgroup. 
To simplify, let us restrict our attention here to subgroups of N . 
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Theorem 1.4. (cf. 3.4) Let G — SU(2,n) and let H be a closed, connected 
subgroup of N . Then H is a Cartan-decomposition subgroup of G if and only if 

1. H satisfies at least one of the eight conditions in Proposition 4.1; and 

2. H satisfies at least one of the five conditions in Proposition 5.1. 

Theorem 1.5. Let G = SU(2,n) and let H be a closed, connected, nontrivial 
subgroup of N . Then H is not a Cartan-decomposition subgroup of G if and 
only if H belongs to one of the eleven types of subgroups explicitly described in 
Theorem 6.1. 

For subgroups H that are not contained in N , there is no loss of generahty 
in assuming that H C AN (see 7.1), and that H satisfies the additional tech- 
nical condition of being compatible with A (see 7.3). Under these assumptions, 
Theorem 7.4, Proposition 7.6, and Lemma 7.8, taken together, list the possibil- 
ities for H and, in each case, determine whether if is a Cartan-decomposition 
subgroup or not. 

Our results require an effective method to determine whether a subgroup is a 
Cartan-decomposition subgroup or not. This is provided by the Cartan projection. 

Definition 1.6. (Cartan projection) For each element g of G, the Cartan 
decomposition G — KA^K implies that there is an element a of with 
g e KaK . In fact, the element a is unique, so there is a well-defined function 

ji: G ^ A'^ given hy g & K ii{g) K . 

The function /i is continuous and proper (that is, the inverse image of any compact 
set is compact). Some properties of the Cartan projection are discussed in [1] 
and [7]. 

We have fJ^{H) = A'^ if and only if KHK = G. This immediately implies 
that if fJ^{H) = A'^ , then H is a Cartan-decomposition subgroup. Y. Benoist 
and T. Kobayashi proved the deeper statement that, in the general case, H is 
a Cartan-decomposition subgroup if and only if (^{H) comes within a bounded 
distance of every point in . 

Notation 1.7. For subsets U and V of we write t/ pa if there is a 
compact subset C of A, such that U C VG and V CUG . This is an equivalence 
relation. 

Theorem 1.8. (Benoist [1, Prop. 5.1], Kobayashi [8, Thm. 1.1]) A closed, con- 
nected subgroup H of G is a Cartan-decomposition subgroup if and only if ii{H) 
A+. 

Remark 1.9. We may consider S0(2,n) to be the subgroup of SU(2,n) con- 
sisting of the real matrices. Then, because A C S0(2,n), we see that S0(2,n) 
is a Cartan-decomposition subgroup of SU(2,n). More generally, a subgroup of 
S0(2,n) is a Cartan-decomposition subgroup of S0(2,n) if and only if it is a 
Cartan-decomposition subgroup of SU(2, n) . (For example, this follows from the 
fact that the Cartan projection for S0(2,n) is the restriction of the Cartan pro- 
jection for SU(2,n).) Thus, our results generahze those theorems of H. Oh and 
D. Witte [14] that are directed toward S0(2,n). 
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Remark 1.10. One may define a partial order <C on the set of closed, con- 
nected subgroups of G by 

Hi -< H2 if there is a compact subset C of G, such that C CH2C . 

(So if is a Cartan-dccomposition subgroup of G if and only \i G -< H .) We see 
from [1, Prop. 5.1] that Hi -< H2 if and only if there is a compact subset C of A, 
such that ijl{Hi) C fi{H2)C . Thus, it is of interest to calculate fJ^{H) , for each 
subgroup H oi G . Our results solve this problem: for each (closed, connected) 
subgroup H , we give an explicit subset U of , such that n{H) ^ U. For 
the cases where fi{H) ^ , these results are summarized in Tables 1, 2, and 3 
of Section 8, and the subset U is given in a standard form that makes it easy 
to determine whether Hi -< H2. Thus, we determine the order structure of the 
relation and also determine precisely where each subgroup lies in this partial 
order. 

The interest in Cartan-decomposition subgroups is largely due to the follow- 
ing basic observation that, to construct nicely behaved actions on homogeneous 
spaces, one must find subgroups that are not Cartan-decomposition subgroups. 
(See [7, §3] for some historical background on this result.) 

Proposition 1.11. (Calabi-Markus phenomenon, cf. [10, pf. of Thm. A. 1.2]) 

If H is a Cartan-decomposition subgroup of G , then no closed, noncompact sub- 
group of G acts properly on G/H . 

H. Oh and D. Witte [15, 16] used this proposition as a starting point 
to study the existence of tessellations. (A homogeneous space G/H is said to 
have a tessellation if there is a discrete subgroup F of G, such that F acts 
properly on G/H, and r\G/H is compact.) In particular, when n is even, 
they determined exactly which homogeneous spaces S0(2, ri)/H have a tessellation 
(under the assumption that H is connected) . These results depend not only on the 
characterization of Cartan-decomposition subgroups, but also on the calculation of 
IJ>{H) for each subgroup H , and on the maximum possible dimension of subgroups 
with a given image under the Cartan projection. In [4] we use some of the results 
of the current paper to study tessellations of homogeneous spaces of SU(2, n) . 

Here is an outline of the paper. Section 2 describes the notation we use 
to specify elements of SU(2,n). Section 3 recalls some general results on Cartan- 
decomposition subgroups, and defines a representation p. Section 4 determines 
whether H contains large elements with ||p(/i)|| approximately equal to 
Similarly, Section 5 determines whether H contains large elements with ||p(/i)|| 
approximately equal to \\h\\. By combining the calculations of the preceding two 
sections. Section 6 determines which subgroups of N are Cartan-decomposition 
subgroups. Then Section 7 determines which other subgroups of G are Cartan- 
decomposition subgroups. Section 8 determines the maximum possible dimension 
of a subgroup of H with any given image under the Cartan projection. 
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from the National Science Foundation (DMS-9801136). Much of the work was 
carried out during productive visits to the University of Bielefeld (Germany) and 
the Isaac Newton Institute for Mathematical Sciences (Cambridge, U.K.). We 
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2 Explicit coordinates in SU(2,n) 



Notation 2.1. We realize SU(2,n) as isometrics of the indefinite Hermitian 
form 

n 

{v \w) = ViW;^ + V2W^ + ^ ViWi + Vn+lW^ + Vn+2Wl 

1=3 

on C"+^. The virtue of this particular realization is that we may choose A to 
consist of the diagonal matrices in SU(2, n) that have nonnegative real entries, 
and to consist of the upper-triangular matrices in SU(2,n) with only I's on 
the diagonal. Thus, the Lie algebra of AN is 



a + n 



' (ti (j) X t] ix\ 

t2 y iy -f) 

-yt -x^ 

-^2 -0 

\0 -tij 



,x, y G C 
x,y G 



Ti-2 



(2.1) 



where or 77 denotes the conjugate of a complex number (f) or rj, and or 
denotes the conjugate-transpose of a row vector x or y. Note that the first two 
rows of any element of -|- n are sufficient to determine the entire matrix. 

Notation 2.2. Because the exponential map is a diffeomorphism from n to A^, 
each element of N has a unique representation in the form expu with u & n. Thus, 
each element h of N determines corresponding values of , x, y, 77 , x and y (with 
tj^ — t2 — 0). We write 

<Ph,Xh,yh,Vh,^h,yh 

for these values. 

Notation 2.3. We let a and P be the simple real roots of SU(2, n) , defined 
by a{a) =01/02 and /3(o) = 02, for an element o of ^4 of the form 

o = diag(oi, 02, 1, 1, . . . , 1, 1, o^\ 07^). 

Thus, 

• the root space Uq, is the 0-subspace in n, 

• the root space Up is the y-subspace in n, 

• the root space Ua+p is the x-subspace in n, 

• the root space Ua+2/3 is the r^-subspace in n, 

• the root space U2/3 is the y-subspace in n, and 

• the root space U2Q+2/3 is the x-subspace in n. 
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Notation 2.4. For a given Lie algebra f) C n, we use 3 to denote f) fl {Ua+2/3 + 

U2q;+2/3 + U2/3) ■ In other words, 



{u e 



and Xu = Vu = 0}. 



(We remark that if 0^ = for every u E i), then [f), f)] C 3 and 3 is contained in 
the center of 1^ .) 

Notation 2.5. For h e SU(2, n) , define 

A{h) = det f . 

V'i2,n+l 'i2,n+2 / 

The following results collect some straightforward calculations that will be 
used repeatedly throughout the paper. 

Remark 2.6. For 



u — 



we have 



exp(M) 



/o 


4> X 


V 


ix \ 





y 


iy 


-V 








-y' 















VO 








/ 








T] - 


x + i 


(py ^ 




1 


y 




iy- 





Id 














] 










( 



e n 



and 



h — exp u & N , 



"2 Fl 



+ i (> 



;i0py + ilm(0xyt)) 



\yx^ - \i4)y + I 

-0 
1 



and 



-|r/|2 +xy - + i|xyt|2 _ i|y|2Rc(77^) 

A(/,) = - |ylm(xi/t0) + ^y2|0|2 _ ^l^n^p 

When = 0, these simplify to: 



exp(ii) = 



/l 





X 


V 


- \^y^ 


• 11 12 \ 

IX — 1 x p 





1 


y 


iy 


-M 


-77 - 








Id 




-y' 















1 


















1 / 
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and 



Similarly, when y = 0, we have 



and 



exp(ii) 



1 (p X 


V + 


1 1^12 
2 1 1 

+ i (x 


- Re(0?7)\ 

- ll<^Py) 


1 


iy 


-V 


- |^0y 


Id 












1 






VO 







1 / 




i0iy - 1^ 




ll-ry)- 



(2.2) 



Remark 2.7. For 



X 77 ix 
= I y iy —fj 



(f) X Tj ix 
and \ y «y I > (2.3) 



we have 



[u,u] 



^0 (f)y — (f)y —xy^ + xy^ + i^y — i(j)y —2i Iin(xx''^ + (f)fj — (pr]y 
—2i Im(yy^) yx^ — yx^ + itpy — i^y 



and 



'000 -{(py - ^y)y^ + 2i^lm{yy^) *^ 

=(00 



(2.4) 



3 Preliminaries on Cartan-decomposition subgroups 



Notation 3.1. We employ the usual Big Oh and little oh notation: for func- 
tions /i, /2 on H , and a subset Z of if, we say /i = 0(/2) for z & Z ii there is 
a constant C, such that, for all large z E Z , we have ||/i(-z)|| < C||/2(2;)||. (The 
values of each fi are assumed to belong to some finite-dimensional normed vector 
space, typically either C or a space of complex matrices. Which particular norm 
is used does not matter, because all norms arc equivalent up to a bounded factor.) 
We say /i = o(/2) for z E Z ii ||/i(-2)||/||/2(-2)|| — > as z — > 00. Also, we write 
/i X /2 if A = 0(/2) and/2 = 0(/i). 
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Definition 3.2. Define p: SU(2,n) GL(C"+2 AC"+2) by = /lA/i, so p 
is the second exterior power of the standard representation of SU(2, n) . Thus, we 
may define ||p(/i)|| to be the maximum absolute value among the determinants of 
all the 2x2 submatrices of the matrix h . 

We now introduce convenient notation for describing the image of a sub- 
group under the Cartan projection /i. 

Notation 3.3. For functions /i, /2 : ^ IR"^ , and a subgroup H of SU(2, n) , 
we write ~ [/i(ll^ll)) /2(||/i||)] if, for every sufficiently large C > 1, we have 

^i(H) ^{aeA-^\ C-Vi(||a||) < ||p(a)|| < CM\\a\\) } . 

(If /i and /2 are monomials, or other very tame functions, then it does not matter 
which particular norm is used.) 

We have A'^ — {a e A \ ai^i > 02,2 > 1 } , so, for a e A'^ , we have 
||«|| = < ai,ia2,2 = < ai,i = 

Thus , so, from Theorem 1.8, we see that if is a Cartan- 

decomposition subgroup of G if and only if ii{H) ^ [\\h\\, \\h\\^^ . This observation, 
which is essentially due to Y. Benoist (in a much more general context, cf. [1, 
Lem. 2.4]), leads to the following result. 

Proposition 3.4. (cf. [14, Prop. 3.24]) A dosed, connected subgroup H of 
SU(2,n) is a Cartan- decomposition subgroup if and only if 

1. there is a sequence {hm} in H , such that hm ^ 00 as n — > 00, and 
p{hm) ^ \\hmf; and 

2. there is a sequence {hm} in H, such that hm ^ 00 as n ^ 00, and 

p{hm) X hm- 

The following result allows us to replace if by a conjugate subgroup when- 
ever it is convenient. 

Lemma 3.5. (cf. [1, Prop. 1.5], [8, Cor. 3.5]) Let H he any closed, connected 
subgroup of SU(2,n) . For every g & G , we have /j{g^^Hg) ^ p{H) . 

In particular, H is a Cartan- decomposition subgroup if and only if g~^Hg 
is a Cartan-decomposition subgroup. 

4 When is the size of p{h) quadratic? 

In this section, Proposition 4.1 is a list of subgroups that contain a sequence 
{hm} with p{hm) X ||/im|P; ^ud Proposition 4.3 is a list of subgroups that do not 
contain such a sequence. Then Proposition 4.4 shows that both hsts are complete. 
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Proposition 4.1. Assume that G — SU(2,n). Let H be a closed, connected 
subgroup of N . There is a sequence /i^ — > oo in H with p{hm) ^ ll^mlP if either 

1. there is an element u of [) with 0^ = 0, such that the vectors and yu are 
linearly independent over C; or 

2. there is an element z of i, such that \r]z\'^ ^ ^ziz', or 

3. there are elements u of \) and z of i, such that = 0, and x^li/^p + 
"izW? + 2 Im(a;„?/1;r^) 7^ 0; or 

^. there is an element u of i), such that 0« 7^ 0, |/„ = 0, y„ = 0, and 
\xu\'^ + 2Re{(l)Ju) = 0; or 

5- U2a+2/3 C f) and there is an element u of i:), such that 0^ 7^ 0, 7^ 0, and 
yu = 0; or 

6. there are elements u and v of i:), such that 0„ 7^ 0, 7^ 0, (py — 0, y^ — 0, 
7^ 0; = 0; and x^y^ — 0; or 

^2a+2i3 C 3; and there are nonzero elements u and v of i), satisfying 0„ 7^ 0, 
Vu 7^ 0, 0„ = 0, y„ = 0, y^ 7^ 0, and x^yl = -i0«y^; or 

8. dim[) = 3, 3 = U2a+2/3, there exist u,v e J) \3, such that |/„ 7^ 0, y^ = 0, 
y„ = 0, Ixy]"^ + 2Re(0^f/^) > 0, and we have 0^ 7^ for every h & f) \3- 



Remark 4.2. In Conclusions (6) and (7), the restriction on Xyyl is not nec- 
essary; it was included to avoid overlap with Conclusion (2). Namely, if x^yl 7^ 
— i0„y„, then [u,v] satisfies y = and rj ^ 0, so Conclusion (2) holds. Also, it is 
not necessary to assume y^ 7^ in Conclusion (7), because Conclusion (6) holds if 
y„ = (and 7^ 0). Thus, (6) and (7) may be replaced with the following: 

(6*) there are elements u and v of f), such that 0„ 7^ 0, |/„ 7^ 0, 0^ = 0, y^ — 0, 
Xy ^ 0, and y^ = 0; or 

(7*) U2Q,+2fl C 3, and there arc nonzero elements u and v oi i), satisfying 0„ 7^ 0, 
Vu 7^ 0, 0^ = 0, yy = 0, and Xy 7^ 0. 



Proof. We separately consider each of the eight cases in the statement of the 
proposition. 

(1) Let /i* = exp(tit). Replacing if by a conjugate under Ua, we may 
assume that Xu is orthogonal to y„; that is, = 0. Then it is clear that 
p(/i*)xA(/i*)xt4x||/i*f. 

(2) Let /i* = exp(i^) . We have /i* x t and 

A(/i*) = xt.ytz -hz\'^^ t^i^zyz -\r]z\^)-t^. 

Therefore x A(/i*) x x 

(3) For any large t, let h = expitu+t"^ z) . Clearly, we have + = 0{t) 
and \xh\ + \yh\ + \Vh\ = 0{t^) , so h ^ 0{t^) . 
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We have 

"1 



ImA(/i*) = 



Therefore, p{h^) x x 

(4) For any large t, let h = exp(t-u). Then /ii,n+2 = ity-u, so it is easy to 
see that /i x t. We have p{h) x x \\h\\^ . 

(5) Replacing by a conjugate (under a diagonal matrix), we may assume 
that (pu — Vu- Then, by renormalizing, we may assume that 4>u — — ^- Let z 
be the element of U2Q+2/3 with = 1 . By subtracting a multiple of z from u , we 
may assume = 0. For any large t, let h = exp(6tM + 36t^^), so /ii,n+2 is real. 
We have 

Re A(/i) = (36t=')(6t) + ^(6^)^(6^)2 + 0{e) x t\ 

so p{h) xt^x \\hf. 

(6) For each large t, let /i be an element of exp(t-u + ) , such that /ii,n+2 
is pure imaginary. (This exists because the sign of — is opposite that of 

We note that Xh x and \r\h\ + — O(i^), but (l)h >^ Vh ^ t and 
IVftl + \xhyl\ = 0(^)- Thus /i = 0{t^) and 

p(/i) X ReA{h) = - Yiil^/'^l'l'^'^l' + 0{t') X X 

(7) Because Xy-yl = —i(f)uyv, we have 7^ 0, so, for any large t, we may 
choose h e exp(tM + M^; + U2q,+2/3) , such that /ii,„+2 = 0. Thus 4>h yh t , but 

>^ Yh >^ and |77^| + |x/i| = O(f^). Then (because /ii,n+2 = 0) it is easy to 
verify that h — 0{t^). However 

Im A(/l) = ly,|0,|2|^,|2 + iy,|a;,|2 + 0(^5) X 

So p(/.)x||/.||2. 

(8) For any large t, choose s = 0(1), such that Re(exp{su + tv)i^n+2) — 0. 
(This is possible, because — ||a;^p — Re^cf)^!}^) < 0.) Then we may choose h G 
exp(s?7, + + 3), such that /ii,„+2 = 0. Then (p^ x t, \xh\ + \r)h\ — 0{t), and 
\yh\ + \yh\ = so we have p{h) x x '"^"^ 



Proposition 4.3. Assume that G = SU(2,n). Let H be a closed, connected, 
nontrivial subgroup of N . 

1. If dimf) = 1, t) = 3, and we have \r]h\'^ = y-hYh for every h e H, then 
p{h) X h for every h E H . 

2. If (ph = Q (ind yh = for every h E i) , 3 C U2Q+2/3 , and there is some m G f) , 
such that Yu 7^ 0, then n{H) ^ ||^||^^^] , unless dimi7 = 1, in which 
case p{h) x ||/?.||'^''^ for every h E H . 

3. Suppose (ph = for every h E i) , and there is some X E C, such that 

— ^yh for every h E H , and we have rjz = iXyz and x^ = jApy^ for every 
zEl- 
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(a) If there is some w e I), such that x„ + |Apy„ + 2Im(Ar7„) 7^ 0; ^hen 
H{H) ^ [\\h\\, ; unless dimH^l, in which case p{h) x \\hf/'^ 
for every h & H . 

(b) Otherwise, p{h) x h for every h & H . 

4. If yu = 0, = 0, and [x^p + 2 Re(0/,7^) ^ /or every hei)\ U2a+2p {so 
3 C U2a+2^), ^/ien X h for every h & H . 

5. If I = 0, there is some e and some nonzero 00 G such that 
(f)u ^ 0, and we have (ph — (poVh o-nd yh — 0, for every /i e f), then 
li{H) ^ [\\h\\,\\h\\^/^], unless dim// = 1, in which case, p{h) x ||/i||V3 
for every h & H . 

6. If dimi) < 3, ^ — 0, we have (py x y^ and v — 0(\(l)y\ + \yv\) for every 

e f), and there exists u E \j, such that 0^ 7^ 0, then p{h) x for 
every h & H . 

7. If dim f) = 2 , 3 = U2a+2^ ; O'^d (^h ^ and y/i 7^ for every h & f) \ 3 ; then 

t,{H)^[\\h\\,\\hr/']. 

Proof. We separately consider each of the seven cases in the statement of the 
proposition. 

(1) Because A(/i) = for every h E H , it is clear that x /i for every 

heH. 

(2) We have \r)h\ + \yh\ = 0{xh), so /ii,n+2 x + |x/j| and hij = 0{xh) = 
0(|/ii,n+2r/') whenever ^ (l,n + 2). Thus, p{h) = O {\\hf/^) . 

We have p{exp{tu)) x Im A(exp(tM)) x x || exp(tM)||^/^. If dimi/ > 1, 
then there is some nonzero v E i), such that = 0. Then, for h e exp(Ri>), we 
have p{h) x Ix^l"^ + \xh\ x h. 

(3) Replacing H hy a conjugate under Ua, we may assume that A = 0, 
so Xh = for every h & H, and r;^ = = for every 2; e 3 (which means 
3 C U2/3). Therefore, the Weyl reflection corresponding to the root a conjugates i) 
to a subalgebra either of type (2) or of type (4), depending on whether or not there 
is some m G f), such that x^ + |Apy„ + 2 lm{Xr}^) ^ 0. 

(4) By assumption, the quadratic form + 2 Re(0?j) is definite on [)/3, so 
\x^ + I0P + I^P = 0(|a;p + 2Re(0?j)) . Therefore, hij = 0(|/?4.„+2|^^^) whenever 

7^ (l,n + 2). Furthermore, hij = 0(1) whenever i ^ I and j 7^ n + 2. Thus, 
p{h) X h. 

(5) For any sequence {km} ^ 00 in i/, we write 0^, x^, y^, y„i, rjm, x-m for 
> etc. 

We have 0„ x y„. If = 0(|ym|^/^), then p{hra) x ReA(/im) x 
Ym ll^mll''^^- (This completes the proof if dim// = 1-) If |ym|^''^ = o{xm), 
then km X /ii,n+2 - |a;mP, but hij = 0(1x^1 + Ym) = 0[\xm\'^/^) whenever 
(hj) (1) ^ + 2) , and hij — 0(ym) — 0(1x^1^/^) whenever i 7^ 1 and j n + 2. 
Therefore 

pihm) 0(|Xni| l^^nil ^ ~\~ \XjYi\ ^ l-^ml ) 0(|Xni| ^ ) 0(||/lm|| ^ ). 

If dim// > 1, then there is some (large) /i G // with y/i = (and hence 
(t)h = 0). Thus p(/i) X X /i. 
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(6) For any sequence {hm} — > oo in iJ, we show that p{hm) x ^{hm) ^ 
ll^mll^^^- We write (t>m,Xm,ym,ym,'nm,^ for , etc. 

If y„ = o(02j, then hi,n+2 >^ ^1, but = 0(0^) whenever 7^ 
(l,n + 2), and hij = 0{4>'^) whenever i ^ 1 and j ^ n + 2. Thus, p{hm) ^ 
ReA(M^0^ x'||/i„||3/2. 

We may now assume that 0^ = 0(ym)- Thus, there is some w G f), such 
that (f)v — and = 1. (Note that, because x 0^, we have y„ = 0.) Because 
[li, t;] e 3 = , we must have r)[u,v] = , so x^yl — —i(t>u)/v 7^ . In particular, 

■^v 7^ , SO Xra Ym ■ 

Wc have = 0{\xm\^) = O(y^), but hij = O(|0my„t| + |ym|) = 

0{\ym\^^^) whenever 7^ (1,72 + 2), and hij = 0(ym) whenever 1 and 

+ Thus, hm = 0{yl) and p(/i^) = O(y^) . 
Furthermore, we have 



lmA{hm) = ^Yml^ml IVml + -ym\Xm\ + ^(y^^m) X yj 



because Yml^^mP x y^, and the terms ^ym|0mPI?/mP and |ym|3^mP cannot cancel 
(since they both have the same sign as y^). We conclude that p{hm) x A(/i^) x 



y3 . 

Jm 



All that remains is to show y^ = 0{hm) ■ li (p^^ o{ym) , then 

1 

"2' 



Re/ii,„+2 = -l\x\^ + 0{<Ply) X yl, 



as desired. If y^ = o(0^) , then 

ReV+2X 0(0^) + 0(0^) + 10:^1x0^, 

so ym — o{(f)m) — 0(0^) = o{hm) , as desired. Thus, we may assume that y^ x (pl 
Because x^ = ym^v + 0{(f)m) and Xyy}^ = -i(pmyv = -«0m, we have 



Im(/ii_„+2) = O(y^) - i|0„py„ 



^ lm(0„(y„a;^)l/^) + 0(0^) 



-'-^mlV - l\(pm\''ym + O(0^) X y^. 



6' 

as desired. 

(7) For 2; e 3, we have p{z) x z. For e [) \ 3 with 7^ 0, we have 
p{exp{tu)) X X ||exp(fw)||3/2 All that remains is to show p{h) = 
for every h & H . 

Note that (ph x yh, and + |r7/i| + \yh\ = 0{(j)h) ■ If (ph = 0(1), 
then it is obvious that p{h) x h. Thus, we may assume \(ph\ 00. Then, 
because x \(ph\^\yh\'^ x 0^, but hij = 0{(ph\yh\'^) = 0{(pl) whenever 

(hj) (1)^ + 2), and h^j — 0{(p1) whenever i 7^ 1 and j 7^ n + 2, we have 

Pih) =o[\m'\<Ph\' + ilM'Y] = o{\M') = 0(1^+21=^/^) = oiwhf/'). m 

Proposition 4.4. Assume that G = SU(2,n). Let H be a closed, connected, 
nontrivial subgroup of N . 

1. There is a sequence /i^ — > 00 in H with p{hm) x if and only if H is 
one of the subgroups described in Proposition 4.1. 

2. There is not a sequence h^ ^ 00 in H with p{hm) x if (ind only if 
H is one of the subgroups described in Proposition 4.3. 
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Proof. It suffices to sfiow tliat H is described in eitfier Proposition 4.1 or 
Proposition 4.3. 

We may assume 

= x^y^ for every ^ e 3 (4.1) 

(otfierwise, 4.1(2) fiolds). Because \r]\'^ — xy is a quadratic form of signature (3, 1) 
on U2/3 + Uo+2/3 + ^2a+2/3, then we must have dim 3 < 1. Thus, we may assume 
f) 7^ 3 (otherwise 4.3(1) holds). 

Case 1. Assume (f)h = and yn = for every h E H (and f) 7^ 3). We may 
assume y^ = for every ;2 G 3, for, otherwise, 4.1(3) holds. Then, from Eq. (4.1), 
we have rj^ — for every 2; G 3. Thus, 3 C U2q,+2/3- We may assume y/i = 
for every /i e if, for otherwise Conclusion 4.3(2) holds. We conclude that 4.3(4) 
holds. 

Case 2. Assume (ph — for every h & H , and there is some u & i) with y„ 7^ 0. 
We may assume that Xh and yt are linearly dependent over C for every h E H 
(otherwise 4.1(1) holds). In particular, there exists A e C, such that Xu — Ay„. 

Subcase 2.1. Assume 3 = 0. 

Subsubcase 2.1.1. Assume there exists vet), such that either x^ ^ Cyu or 
yv ^ Cyu- We may assume there exists w e f}, such that x^ 7^ \yw (otherwise 
4.3(3) holds). Furthermore, by adding a small linear combination of u and f to -u;, 
we may assume that yw 7^ and that either x^ ^ Cyu or y^ ^ Cyu- Because x^ 
and yy, are linearly dependent, there exists Ai (7^ A) such that x^ = Ai|/^. (Then 
note that we must have y^, ^ Cy„.) Then 

Xu+w = Xu + Xyj = Xyu + Xiyw ^ C{yu + yw) = Cyu+w 

(because A 7^ Ai and {yu,yw} is linearly independent over C). This contradicts 
the fact that and yu+w are linearly dependent over C. 

Subsubcase 2.1.2. Assume Xh,yh ^ ^yu, for every h E i) . For each h E i), there 
exist A^.,Ay E C, such that x^ = Xxyu and yh = Xyyu- Because 3 = 0, we must 
have yih^u] = , so Im{yhyD = , which means that A^^ is real. We must also have 

Vlh,u] = 0, so 

= -Xhyi + Xuyl = (-A^ + XXy)\yu\'^ = (-A^ + XXy)\yX. 
Thus Aj; = XXy , so 

Xh = KVu = AAj,yu = Xyh- 

Therefore 4.3(3) holds. 

Subcase 2.2. Assume 3 7^ 0. We show that either 4.1(2), 4.1(3) or 4.3(3) holds. 
Straightforward calculations show that conditions 4.1(2), 4.1(3) and 4.3(3) are 
invariant under conjugation by Ua, so we may assume that A = 0; that is, Xu — 0- 
Thus, we may assume = for every -263, for, otherwise, 4.1(3) holds. Then 
we may assume rjz = for every z E for, otherwise, 4.1(2) holds; therefore 
3 = f^2/3- We may now assume Xh = for every /i G I), for, otherwise, 4.1(3) holds. 
Thus, 4.3(3) holds (with A = 0). 
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Case 3. Assume there exists u ^ \) with 0„ 7^ 0. We claim that 3 C U2a+2/3- H 
not, then there is some e 3, such that either ry^ 7^ or 7^ 0. If y^; = 0, 
then |?72p 7^ = x^^y^^, so 4.1(2) holds. On the other hand, if y^ 7^ 0, then, letting 
z' — [m, z] , we have y^/ = and ry^/ 7^ 0, so 4.1(2) holds once again. 

Subcase 3.1. Assume yh = for every h G t) . We may assume that there is some 
w G f), such that y^ 7^ (otherwise, either 4.1(4) or 4.3(4) holds). Then we may 
assume 3 = (otherwise, 4.1(5) holds). 

We claim that 4.3(5) holds. If not, then there is some w E I), such that 
(t)w ^ ^ and y«, = 0. Then 77[„,«,] 7^ 0, which contradicts the assumption that 



Subcase 3.2. Assume there is some v E\), such that y„ 7^ 0. 

Subsubcase 3.2.1. Assume 3 = U2a+2/3- Suppose, for the moment, that there exists 
w E with (pyj — 0. We may assume that yw — ^ (otherwise, 4.1(3) holds). 
Therefore Xu, ^ 0, so 4.1(7*) holds. 

We may now assume that 7^ for every w G I) \ 3 . This implies that x , 
y, ?7, and y are functions of 0; in particular, dimf) < 3. Also, because 3 7^ and 
M, v ^ 3 , we must have dim 1) > 2 . 

We claim dimf) = 2 (so 4.3(7) holds). If not, then dimf) = 3, so there 
exist li, w G , such that 0^ = 1 and (py, — i. Because (p[u,w] = , we must have 
[u,w] G U2a+2/3- Therefore = X[u,w] ^ yw - iyu, so y^j = iyu- Furthermore, 



so i/u = 0. Then y^j = iyu is also 0. This implies yh = for every h G H . This 
contradicts the fact that y^, 7^ 0. 

Subsubcase 3.2.2. Assume 3 = 0. Lemma 4.5 below implies that either 4.3(6) 



Lemma 4.5. Let H be a closed, connected subgroup of N , such that 3 = 0, 
and assume there exist u,v G f) , such that 7^ and yy 0. Then either H 
is described in 4.3(6) {and in 5.2(4), which is the same), or H is a a Cartan- 
decomposition subgroup {and is described in 4.1(6*) and 5.1(2)). 

Proof. Let us begin by establishing that (j)h >^ yh ior h e l) . If not, then we 
may assume either that yu — or that 0„ = 0. Then, because [[u, v],!'] G 3 = 0, 
we see from Eq. (2.4) that 

= -{(f>uyv - (t>vyu)yl + 2i0^ Im(y„y|) = -(t>u\yv? - + 7^ 0. 
This contradiction establishes the claim. 

Case 1. Assume there is a nonzero w E ^, such that 0^ = and y^ = 0. Note, 
from the preceding paragraph, that = . Then, because 3 = , we must have 
x^ ^ 0. Therefore, 4.1(6*) and 5.1(2) hold, so n{H) ^ \\\h\l\\hf], so // is a 
Cartan-decomposition subgroup. 

Case 2. Assume there does not exist such an element w G 1^ . Then H is described 



3 = 0. 



= )/[uM = -2i Im(y„y^) = -2ilm(-i|?/„p) = -2i|y, 




in 4.3(6) and in 5.2(4). 
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5 When is the size of p{h) linear? 

In this section, Proposition 5.1 is a Hst of subgroups that contain a sequence 
{hm} with p{hm) >^ hm, and Proposition 5.2 is a hst of subgroups that do not 
contain such a sequence. Then Proposition 5.3 shows that both hsts are complete. 

Proposition 5.1. Assume that G = SU(2,n). Let H be a closed, connected 
subgroup of N . There is a sequence km ^ oo in H with p{hm) ^ hm if either 

1. there is a nonzero element z of i with \riz\'^ = x^^y^; or 

2. there is an element u of i), such that 4>u — 0, dimc{x,y) — 1, and 

Xt,|y„|^ + y„|x„|^ + 2 Im{xuylrh) = 0; 

or 

3. there is an element h of H with yh = 0, y/i = and |a;/ip + 2 Re(0/i?]/^) 7^ 0; 
or 

4. there are elements u of ^ and z of i, such that 0„ 7^ 0, = 0, yu 7^ 0, 
Tjz ^ Q , and y^ = 0; or 

5. there are nonzero elements u of \) and z of i, such that 0^ 7^ 0, yu ^ 0, 
yz = 0, <j>u>h is real, and 

^zlyul"^ - KyuWz + 2im(^x„i/y = 0. 

Proof. We separately consider each of the five cases in the statement of the 
proposition. 

(1) From 4.3(1), we have p{h) x h for all h G cxp(]Rz). 

(2) Replacing iJ by a conjugate under {Ua,U^a), we may assume that 
T/u = (and Xu 7^ 0). Then, from the assumption of this case, we know that y^ is 
also 0. Therefore, 4.3(4) implies that p{h) x h for all h e exp(RM). 

(3) Prom 4.3(4), we have p{h) x h for aU h e exp(Rit). 

(4) . For any large t, choose h G cxp(t-u + 3), such that xtyh + j^^l^/iPvl — 
\r]h\^ = 0. Note that r]h x {(phVhl t"^ , so h ^ but h,j = 
0{t^) whenever (i,j) 7^ (1,72 + 2), and /ijj = 0{t) whenever i ^ {1,2} or 
J ^ {n + 1, n + 2} . From the choice of h, wc have 

A(/i) =0 + 2 iWx^^'yh) = 0{e) = Oih), 

so it is not difficult to see that p{h) x h. 

(5) Replacing [) by a conjugate, we may assume u G u„ + u^g. (First, 
conjugate by an element of t/g to make y^ = 0. Then conjugate by an element 
of Ua to make Xu orthogonal to y„. Then conjugate by an element of C/g that 
centralizes yu, to make Xu — 0- Then conjugate by an element of Ua+p to make 
r)u = 0. Then conjugate by an clement of Ua+213 to make x„ = 0.) Then, by 
assumption, we must have = 0, because y« = and a:„ = 0. 

Furthermore, replacing () by a conjugate under a diagonal matrix (that 
belongs to G), we may assume that 0^ and yu are real. Then rjz must also be 
real (because 0„77^ is real). Thus, we see that u,zE so(2, n) . So [14, Thm. 5.3(1)] 
implies that if is a Cartan-decomposition subgroup. ■ 
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Proposition 5.2. Assume that G — SU(2,n). Let H be a closed, connected, 
nontrivial subgroup of N such that 

\Vz\'^ 7^ XzYzj for every nonzero 2; e 3. (5.1) 

1. // = 3 (so dim if < 3), then p{h) x \\h\\^ for every h e H . 

2. If (ph — and dimc{xu, yu) 1 fof^ every h Ei), then p{h) x for every 
heH. 

3. If (ph = for every /t e f), there exist nonzero u and v in i) , such that 
dimc(x„,y„) ^ 1 and dimc{x^,yy) = 1, and x^\y^\'^+y^\xy\^+2lm{x^ylr]:;;) ^ 
for every such v ei), then IJ.{H) . 

4. If dim [) < 3, ^ = 0, we have (pv x y^ and v = 0{\(})y\ + ly^,!) for every 
V E i) , and there exists w e f), such that 0^ 7^ 0, then p{h) x for 
every h E H . 

5. If dimt) < 2 and 4>h ^ 0, yh = 0, yt = 0, and |a;/jp + 2Re{4>hf]h) = for 
every nonzero h Ei), then p{h) x for every h E H . 

6. If dimf) = 2 and there exist nonzero u E \) and z E i, such that 0„ 7^ 0, 
l/u 7^ 0, = 0, (pjj; is real, and x^||/„|2 - 0yy„r]J + 2 Im (?7Jx„yi) ^ 0, then 
p{H)^[\\hfl\\\hf]. 

1. If dimf) = \, and we have 4>h = 0, dimc{xh, yh) = 1; and 

^h\yh\^ + Vh\xh\^ + 2 lm(xhylfih> 7^ 
for every nonzero h Ei:), then p{h) x for every h E H . 

8. If dim = 1, and (ph 7^ 0, yh = 0, and yh 7^ 0, for every nonzero h E i), 
then p{h) x for every h E H . 

Proof. We separately consider each of the eight cases in the statement of the 
proposition. 

(1) From Eq. (5.1), we know that the quadratic form Iryp — xy is anisotropic 
on 3 = f), so 

A{h) = \7]h? - Xhyh X \r]h\^ + + y^ x \\h\\\ 

(2) Because dimc{xu,yu) 7^ 1, we have 

I |2| |2 I t|2 „ I |4 I I |4 

\xh\ \yh\ - \xhyh\ ^ \xh\ + \yh\ , 

so Lemma 5.4 imphes p{h) x 

(7) From either Proposition 4.3(2) or 4.3 (3a) (depending on whether yh is 
or not), we have p{h) x for every h E H . 

(3) From Lemma 5.4, we have H/i-P^^ = 0[p{h)) . 
From (2), we see that p{h) x for h E exp(MM). 
From (7), we see that p{h) x \\h\\^/^ for h E exp(Mt'). 

(4) See Proposition 4.3(6). 

(5) Because Rehi^n+2 = 0, it is easy to see that p{h) x 0^ x 
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(6) Replacing if by a conjugate, we may assume — and = 0. 
Therefore, Xh — and = for every h e H . Thus 

XzlVul'^ = ^zlVul"^ - 0«yu^+ 2Im(%a;„2/y 7^ 0, 

so Xz 7^ 0. From Eq. (5.1), we know rjz 7^ 0. 
We have p{tz) x \\tzf (see 5.2(1)). 

Because 0^ is a real multiple of riz , we may let h he a large element of H , 
such that f]h = -\yh\'^4>h/'^'^ + 0{4>h)- (So x (ph and x/j x 77/^ x 0^.) Then 

= O(0t) + ^Qx,||/,p) X05,. 
It is clear that all other matrix entries of p{h) are 0(0^). Thus, we have p{h) x 

Now suppose there is a sequence h„i ^ 00 in H with p{hm) = • 
Case i. Assume rjm = o(0^) . We have /i^ ^ 4>tm 

<tt X Re A(/i^) = 0(p(/i„)) = o(||/i^||5/^) = o{<t>l). 
This is a contradiction. 

Case 2. Assume 0^ = o{r]m) ■ We have /i^ x Re/ii_„+2 x (prnVm, so 

77^ X Re A(0 = = o(||/i„||5/^) = o(||/i^||^/^) = 0(10^77^1^/2) = 0(77^). 

This is a contradiction. 

Case 3. Assume rjm x 0^- We have hm = 0(0^) , so 

0^ X x^ly^l' X Im A(/i„) = 0(p(/ij) = o(||/i„,||'/^) = o(0^). 

This is a contradiction. 

(8) See Proposition 4.3(5). ■ 

Proposition 5.3. Assume that G — SU(2,7i). Let H be a closed, connected, 
nontrivial subgroup of N . 

1. There is a sequence /i^ — > 00 in H with p{hm) x km if and only if H is 
one of the subgroups described in Proposition 5.1. 

2. There is not a sequence /i^ — > 00 in H with p{hm) ^ ||^m||^ if and only if 
H is one of the subgroups described in Proposition 5.2. 

Proof. It suffices to show that H is described in either Proposition 5.1 or 
Proposition 5.2. 

We may assume (5.1) holds (otherwise. Conclusion 5.1(1) holds). 

Case 1. Assume 0/i = for every h E H . We may assume there exists v G t), 
such that dimc{xv,yy) = 1 (otherwise 5.2(2) holds). Furthermore, we may assume 
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Xi;|?/i;P + yi;k?;P + 2 Im(a;i,|/^?7;;) 7^ for every such v (otherwise 5.1(2) holds). Then 
we may assume dimc(x„, yu) — 1 for every nonzero u & i) (otherwise 5.2(3) holds). 

The argument in Subsubcase 2.1.1 of the proof of Proposition 4.3 implies 
there exists A G C, such that, for every h & H , we have Xh = ^Uh (or vice- 
versa: for every /i, we have Uh = ^x^). Thus, replacing i7 by a conjugate under 
{Ua-i U_a) , we may assume Xh = for every h E H . 

If dim H > 1, then there is some nonzero u E i), such that = . This 
contradicts the fact that y-vlUv]"^ + Vvl^v]'^ + 2 Im(a;^y^77^) 7^ 0. Thus, we conclude 
that dim// = 1, so 5.2(7) holds. 

Case 2. Assume the projection of () to is one-dimensional. Replacing H hy a 
conjugate under A, we may assume 4>h is real for every h E H. Fix some u E i), 
such that 0u 7^ 0. 

We may assume that U2a+2/3 ^ i) (otherwise Conclusion 5.1(1) holds). 
Therefore [i),u] must be zero, so yz = and rjz is a nonzero real, for every 
nonzero 2; e 3. (This implies dim 3 < 1.) 

Subcase 2.1. Assume i/h — for every h E H . We may assume Conclusion 5.1(2) 
does not hold. 

We claim that i) = Wu + 3. Suppose not. Then there is some v E i), such 
that (f)^, = and x^ ^ 0. Because Conclusion 5.1(2) docs not hold, we must have 
Yv 7^ 0- Then [v,u,u] is a nonzero element of U2a+2j3- (This can be seen easily by 
replacing H with a conjugate, so that u E Uq.) This contradicts our assumption 

that U2a+2/3 ^ i) ■ 

If Yu 7^ 0, then either Conclusion 5.2(8) or 5.1(4) holds (depending on 
whether 3 is or not). If y„ 7^ 0, then 5.1(3) or 5.2(6) holds. 

Subcase 2.2. Assume the projection of i) to U/3 is nontrivial. Then we may assume 
Vu^O. 

Subsubcase 2.2.1. Assume there are nonzero v E \) and z E i, such that (f)^ = 0, 
yv = 0, and x^ ^ 0. We may assume that Conclusion 5.1(5) does not hold. 
Therefore, for every real t , we must have 

7^ ^zlVul'^ - 0«(y« + ty^)rh + 2 Im(f7l(x„ + tXy)yl) 
= t [-^'uyv'fh + 2 lui(r];xyyl)] + constant. 

Thus, the coefficient of t must vanish, which (using the fact that rj^ is real and 
nonzero) means 

O = -0«y^ + 2Im(x^y1;). (5.2) 

We have [u,v] G 3 , so ri[u,v] is real. Thus, 

= Im7y[„,„] = lm{xyyl + i(t)uyv) = Im(x„|/y + 0„y^. 

Comparing this with Eq. (5.2), we conclude that 0„y„ = 0. Therefore y„ = 0, so 
Conclusion 5.1(2) holds (for the element v). 

Subsubcase 2.2.2. Assume there do not exist nonzero v E^ and z E such that 
(pv = 0, yv = 0, and x^ ^ 0. We must have 

yw — for every w E I), such that (p^ — 0. (5.3) 
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(Otherwise, we obtain a contradiction by setting v = [u,w] and z — [u,w,w].) 
We may assume 

Yv for every v E i) such that 0^ — 0, — 0, and x-t, ^ 0. (5.4) 

(Otherwise, Conchision 5.1(2) holds.) 

We claim dim[) < 2. If not, then there exist linearly independent v, w G f), 
such that — (pu, — 0- From (5.3), we know that yv — yw — 0- By replacing 
with a linear combination, wc may assume y«, = 0. Then, from (5.4), we know 
that Xy, = 0, so w E ^. Because 3 is (at most) one-dimensional, but v and w 
arc linearly independent, wc know that v ^ ^, so Xy ^ 0. This contradicts the 
assumption of this subsubcase. 

We may now assume dim — 2 (otherwise Conclusion 5.2(5) holds). Choose 
a nonzero v E I), such that 0„ = 0. If x„ 7^ 0, then Conclusion 5.2(5) holds. If 
Xv — 0, then i> e 3, so either Conclusion 5.1(5) or 5.2(6) holds. 

Case 3. Assume the projection of i) to Uq is two-dimensional. We may assume 
3 = (otherwise, U2a+2/3 C f), so Conclusion 5.1(1) holds). We may assume yh — 
for every h E H (otherwise Lemma 4.5 implies that either 5.2(5) or 5.1(2) applies. 
Therefore [[),[)] C 3 = 0, so f) is abelian. 

Let with 0^ = 1 and 4>y = i. Then 

= r][u,v] = iVv + Vu, 

so y„ = y„ = 0. Then, for every w E 1), we have = 'r)lu,w] — iyw, so Yu, = 0. We 
may assume 

\xh\' + 2Re{<Phr]K) = (5.5) 

for every h E I) (otherwise Conclusion 5.1(3) holds). This implies dimf) = 2 
(otherwise, there is some w E i) such that (f)yj = and Xu, 0, and then Eq. (5.5) 
does not hold for h — u + tw when t is sufficiently large). Thus, Conclusion 5.2(5) 
holds. ■ 



Lemma 5.4. Let H be a closed, connected, nontrivial subgroup of N . Assume 

(ph = for every h E i) , that (5.1) holds, and that x^ly^]'^ + yy\xy\'^ + 2lm{xvylr]^) ^ 
for every v E\) such that dimc(a;^, = 1. Then = 0(A(/i)) for every 

hEH. 

Furthermore, A(/i) x whenever \xh\'^\yh\^ — \xhy\}\^ ^ l^hl"^ + b/il"^- 

Proof. We have h x \xh\'^ + \yh\'^ + |x/i| + \yh\ + \Vh\- Also, from Eq. (5.1), we 
have \r)z\'^ - x^y^ x (|x^| + jy^l + |?7^|) for every 2; e 3. Also, x^||/^p + y^|a;^p + 
21m{xyylfh) X jvp whenever dimc{xy,yv) = 1- 

Case 1. Assume \xh\^\yh\^ — \xhyh\'^ = o(|a;fe|^ + {yhl"^) ■ Then there is some e f) 
such that V — log/i — o[\xh\ + \yh\) and |a;^p||/j;p — = 0. We have 
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dim£{xy,yjj) = 1. Therefore 

ImA(/i) X Xh\yh\'^ + yh\xh\'^ + 2lm{xhylWi) 
= x„||/^P + y^lx^P + 2Im(a;„?/i?7;;) 

+ o(|77,|3 + |x,|3 + |y,|3 + |a;,|3+|y,n 
X \vf + o{\r]hf + \xh\^ + \yhf + \xh\^ + \yhf) 

Thus, \\hr/'^0{p(h)). 

Case 2. Assume \xh\'^\yh\'^ — Ixhy^l"^ x Ixhl"^ + \yh\'^ ■ We may assume ReA{h) — 
o(|x/j|^ + for otherwise it is clear that ReA(/i) x (So we have 

ll^ll ^ 1^7/11 + |x/i| + |y/i| ^ \xh\'^ + Thus, there is some e 3, such that 

z — log /i = o(log h) and 

i%r - x.y. = -^(k,rii/,r - 1 wr) + ^(k.r + i^.r) < o. 

(This implies that and must have the same sign.) Prom (5.1), we conclude 
that \rjz\'^ — XzYz < for every 2; e 3. Thus, there is a constant e < 1, such that 
l^zl < ^\/^zYz for every 2; e 3. Then 

|Im(a;/,|/ir^)| < \qz\W\\yh\ < ^\^z\yh\^ + yz\xh\^\, 

so 1111 

Therefore 

1 1 
ImA(/i) = lm{xhyif]i;) + -Xh\yhf + -yh\xh\'^ 

= Im(a:ftylr^) + ^x^ly^l^ + ^y^jx^l^ + o((|a;ft|^ + ly^H log /i) 

X ^x^bftP + ^y^kftl^ 

X Whf. 



6 Non-Cartan-decomposition subgroups contained in N 

Theorem 6.1. Assume that G = SU(2,n). Here is a complete list of the 
closed, connected, nontrivial subgroups H of N , such that H is not a Cartan- 
decomposition subgroup. 

1. If dim I) = 1, {) =3, and we have \rjh\^ = x/^y/j for every h & H, then 
p{h) X h for every h & H . 
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2. If (f)h — and Uh = ^ for every h E i), there is some u E i), such that 

7^ 0, and 3 C U2a+2p, then Ijl{H) [\\h\\, ; unless dimH — 1, in 

which case p{h) x for every h E H . 

3. Suppose (ph = for every h & I), and there is some X & C, such that 

— ^Uh for every h & H , and we have rj^ — iXy^ and = lApy^ for every 

(a) If there is some u & ^, such that + |Apy„ + 2Im(Ar7^) 7^ 0; then 
H{H) ^ [\\h\\, ; unless dimH ^1, in which case p{h) x \\hf'^ 
for every h & H . 

(b) Otherwise, p{h) x h for every h & H . 

4. If Vh = 0, Yh^ 0, and \xh\^ + 2Re(0/,r^) 7^ for every h U2a+2p {so 
3 C U2a+2/3), then p{h) x h for every h & H . 

5. If I = 0, there is some u E i) and some nonzero (po E C, such that 
0u 7^ 0, and we have (t)h — ^oYft and yh — 0, for every h & then 
li{H) ^ [11^11,11^11^/^]; unless dim// = 1, in which case, p{h) x ||/i||V3 
for every h E H . 

6. If (ph — and dimc(xu, y„) 7^ 1 for every h E i), and \r)z\^ ^ ^^zYz, for every 
nonzero z El, then p{h) x for every h E H . 

7. If (ph = for every h E i), there exist nonzero u and v in \), such that 
dimc(x„, 7^ 1 and dimc(x^, = 1; and we have y.v\yv\'^ + YijI^^i^P + 
2 Im(xt,|/Jr/^) 7^ for every such v E'i}, and \r]z\'^ 7^ x^y^, for every nonzero 
z E 3 then p{H) « [\\hfl\ \\hf] . 

8. If diml^ < 3, 3 = 0; u;e have 0„ x y^ and v — + |y^|) for every 
V E \), and there exists u E {\, such that (pu 7^ 0, then p{h) x for 
every h E H . 

9. If dm\\] =2,3 = U2a+2/3, 0fe 7^ and y/i 7^ for every h E then 
^^{H)^[\\h\\,\\hr/']. 

10. If dimf) < 2 and (ph 7^ 0, = 0, y^ = 0, and \xh\^ + 2Re{(f)hf]j;) = for 
every nonzero h Ei), then p{h) x for every h E H . 

11. If dimf) = 2 and there exist nonzero u E ^ and z E i, such that 0„ 7^ 0, 
Vu 7^ 0, y^ = 0, (t)urrz 7^ is real, and x^|y„p - 0„y„77; + 2 Im(77;x„yi) 7^ 0, 
then ^l{H) ^ [\\hr/\ \\hr] . 

Proof. The theorem is obtained by merging the statement of Proposition 4.3 
with the statement of Proposition 5.2, and ehminating some redundancy (see 3.4). 
Specifically: 

• 4.3(1) appears here as 6.1(1). 

• 4.3(2) appears here as 6.1(2). 

• 4.3(3) appears here as 6.1(3). 
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• 4 r{(A) 


^mnpj^r^ Viptp R 1 f 4 1 

CXkJ L/C'Cll O ±ld C CIO IJ. J- I ^ 1 . 




• 4 SfS^l 


j^nriPJ^T^ nPTP j^*^ 1 1 R i 

Cl \IjCHi± O ±ld C ClitD \J . J. \KJ J • 






fmT^Pfl"r<3 nPTP R 1 i R 1 




• 4.3(7) 


^innpi^r^ hprp ^is (S 1 ( Q i 




• 5 2(^) 


1*5 "^TlPPlJ^l PJ^'^IP of f\ 1 1 n 1 
±0 CX iJlJ\J\^±<Ai± kjCmj\J \J1. \J.J.\\JJ. 




• 5 2(2) 


j^'n'npj^T'*^ ViPTP j^<^ R 1 f fl 1 

ClL/L/ Cello ±±d C ClO W.XIWJ. 




• 5 2(?>\ 


i^nnp/^rt; Viprp (\ 1 i 7 i 




• ."i 2('4'l 


i^'n'nPi^T'<5 Iiptp i^<5 R 1 i R i 

d L/ ecu O llCl C dO W . i I (J J . 




• 5 2('5'l 


Piriripj^r^ Iiptp R 1 i 1 1 

Ct L/ v3Cl± O ilC'l v3 CIO . i I i \J J . 




• 5.2(6) 


appears here as 6.1(11). 




• 5.2(7) 


is a special case of 6.1(3a) (with dimH - 


-1) 


• 5.2(8) 


is a special case of 6.1(5) (with dim if = 


1)- 



Corollary 6.2. Assume that G = SU(2,n). Here is a complete list of the 
closed, connected, nontrivial subgroups H of N , such that H is not a Cartan- 
decomposition subgroup, and Na{H) is nontrivial. 

1. Suppose dimf) — 1, \) — I, and we have \r]h\'^ — x^y/j for every h & H . 

(a) Ifi)^ U2/3 or i)^ U2a+2/3, then Na{H) = A. 

(b) Otherwise, Na{H) = ker(Q;) . 

2. Suppose (f)h = and yt = for every /i G f), there is some -u G f), 
such that y„ 7^ 0, and 3 C U2a+2p- U ^ = ^ {'^a+/3 + ^2/3)) + 3; then 
Na{H) =ker(a-/3). 

3. Suppose (f)h = for every h E i) , and there is some nonzero A G C, such 
that Xh = Xyh for every h E H , and we have rjz = iX^z and = jApy^ for 
every z e$. // f) = (f) n (u/3 + u„+/3)) + 3 7^ 3; then Na{H) = keT{a) . 

4. Suppose 4>h = and Xh = for every h E i), we have 3 C U2/3, and t) 7^ 3. 

(a) //^ = (bnu^)+3, thenNA(H)^A. 

(b) Otherwise: 

i. 7/ f) = (f) n (u^ + Ua+2p)) +3; then Na{H) = ker(Q; + (3) . 
a. 7/ f) = ({) n (u^ + U2a+2^)) +3; then Na(H) = ker(2Q; + /3) . 
Hi. 7/3 = and 1^ C + U2p, then Na{H) — ker(/3) . 

5. Suppose yh = 0, yh = 0, and \xh\'^ + 2Re{(j)hr)K) ^ for every h G i)\u2a+2p- 
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(a) // f) = n u,+^) +3, then Na{H) = A. 

(h) Ifi)C Ua+f3 + U2a+2i3, but [) ^ (f) H u^+^g) + 3 , ^/ien Na{H) = ker(a + /5). 

("Cy) //{)=([) n (U„ + Ua+p + Ua+2p)) + I , hut \) U„+/3 + U2a+2/3 , t/iCn 

Ar^(iJ)=ker(/3). 

^. Suppose 3 = 0, there is some nonzero (po E C, such that (ph = (poYh (ind 
Uh = , for every h E i) , and there is some u E i) , such that 0„ 7^ 0. // 
f) = (f) n (u„ + U2;3)) + (J) n Ua+13) , then Na{H) = ker(a - 2/3) . 

7. Suppose (f)h = and dimc{xu,yu) 7^ 1 for every h E i), and \r}z\'^ ^ x^y^, for 
every nonzero z E i- 

(a) // f) C u«+2/3; then Na{H) = A. 

(h) Ifi)<^ Ua+2/3; and = n (u^ + Ua+p)) +3; then Na{H) = ker(Q;) . 

8. Suppose (ph = for every h E i), there exist nonzero u,v E f), such that 
dimc(xu,y„) 1 anc? dimc(x^, y^) = 1, Xy\yy\^ + y^\xy\'^ + 21m{xyylrh) ^ 
/or every such v E i), and \r]z\^ ^ y^z^/z, for every nonzero z E^. 

(a) Ifi) = {i)n{Ua+f3 + U2f3)) + {i)nua+2f3), then N a{H) = k.ei{a - {3) (and 
dim// < 3;. 

(h) // f) = (f) n {up + U2a+2^)) + (f) n u«+2^), then Na{H) = ker(2Q; + (3) 
(and dimi/ < 3 j. 

5. Suppose dimf) < 3, I) = (t)n(u„ + %)) + (l)n(u„+^ + U2/3)) , f)n(Ua+u^) 7^ 0, 
anc? w;e Ziave (ph ^ a'^c? ^^^i x y^, for h E\), then Na{H) — ker(Q; — pi) . 

10. Suppose dimf) = 2, 3 = U2a+2/3, (Ph 7^ and yh ^ for every h E \:)\^. If 
f) = ({) n (u„ + Up)) + 3 , then Na{H) = ker(a - /?) . 

11. Suppose dim f) < 2 an(i (ph ^ 0, yh — 0, yh — 0, and + 2 Re{(phr]h) — 
/or ever?/ nonzero /i e I) . 

fa; // f) C t/ien A^a(/^) = A. 

(b) Ifi)GUa + Ua+/3 + u«+2^; but I) ^ u« , ^/icn Na{H) = ker(/3) . 
fcj // f) C u« + U2a+2/3, , ^/icn Na{H) = ker(Q; + 2/3) . 

Proof. It is clear that each of the given subgroups is normahzed by the indi- 
cated torus. We now show that the hst is complete, and that no larger subtorus 
of A normalizes H . 

Assume Na{H) is nontrivial. We proceed in cases, determined by Theo- 
rem 6.1. 

Case 1. Assume 6.1(1). We may assume f) is neither U2/3 nor U2a+2/3 (otherwise 
(la) apphes). Then, because jr^^p = x„y„ for every it e f), we see that rju for 
every nonzero u E i). Thus, the projection of f) to Ua+2f3 is nontrivial. However, 
because |r7„p = x„y„, we have i)nUa+2i3 = 0. We know that [) C UQ,+2/3+U2/3+U2a+2/3 
(because f) =3), so, because each of 2(3 and 2a + 2(3 differs from a + 2(3 by a, 
we conclude that Na{H) = ker(Q;), so (lb) applies. 
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Case 2. Assume 6.1(2). Let V be the projection of f) to Ua+(3 + U2/3. Because 
y„ 7^ , we know that V projects nontrivially to U2/3 . However, because 3 C U2Q+2/3 , 
we also know that V fl U2/3 = 0. Therefore Na{H) = kei^a — Then, because 
neither a + 2j3 nor 2a + 2/3 differs from a + j3 by a multiple of a — /? , we conclude 
that [) = (t) n (Uq,+^ + U2/3)) + 3, so (2) applies. 

Case 3. Assume 6.1(3). We may assume f) 7^ 3 (otherwise Case 1 applies). 

Subcase 3.1. Assume A 7^ 0. Because f) 7^ 3, the projection of f) to u^j + Uq+^j is 
nontrivial. However, because A 7^ 0, this projection intersects neither U/3 nor Uq,+/3. 
Therefore Na{H) C kcr(a). Then, because neither 2/3, a + 2/3, nor 2a + 2/3 differs 
from /3 by a multiple of a, we conclude that I) = (f) fl (u^ + Ma+p)) + 3, so (3) 
applies. 

Subcase 3.2. Assume A = 0. This means Xu = for every u Ei), and 3 C U2^. 

Because f) 7^ 3, we know that () projects nontrivially to U/3. Because 
3 C U2/3, we know that i) fl Ua+2/3 = ^ fl U2Q+2/3 = 0. Thus, it is easy to see 
that if i) projects nontrivially to Ua+2/3 or U2a+2/3 then either (4(b)i) or (4(b)ii) 
applies. 

Thus, we may assume f) C + U2/3. If 3 7^ 0, then f) = ([) n u^j) + U2/3, so 
(4a) applies. Otherwise, (4(b)iii) applies. 

Case 4- Assume 6.1(4). 

Subcase 4-1- Assume the projection of f) to is trivial. Because 

\xu? = \xu\^ + 2Re(0ur/;;) 7^ 

for every m e I) \ U2a+2/3 , we know that 7^ for every u E i)\ U2a+2i3 ■ Thus, if 
the projection of f) to Uq+2/3 is nontrivial, then Na{H) = ker(/3), and we see that 
(5c) applies. If not, then f) C Ua+p + vi2a+2/3, so either (5a) or (5b) apphes. 

Subcase 4-2. Assume the projection of f) to Uq, is nontrivial. Let y be the 
projection of f) to Uq + Ua+p + Ua+2/3- Because |a;up + 2Re(0„r/^) 7^ for every 
e I) \ U2a+2/3, we know that V (lUa — 0. Then, because a, a + and a + 2/3 
all differ by multiples of /3, we conclude that Na{H) — ker(/3). Therefore (5c) 
applies. 

Case 5. Assume 6.1(5). Let V be the projection of f) to + U2/3. Because (f)h — 
0oy/j, we see that VHUa — O and V fl U2/3 = 0. Therefore Na{H) = keT{a — 2f3) . 

Because no other roots differ by a multiple of a — 2/3 (and 3 = 0), we 
conclude that f) = (f) n (Uq, + U2/3)) + (f) n Ua+p) ■ Thus, (6) apphes. 

Case 6. Assume 6.1(6). 

Subcase 6.1. Assume 7^ 3. Let V be the projection of \) to Uf)-\-Ua+p. Prom the 
assumption of this subcase, we know 1/ 7^ 0. However, because Amic{xu,y^ 7^ 1 
for every m G I), we know that V Ujj — and () fl Ua+/j = 0. Therefore 

Na{H) = ker(Q;) , so (7b) applies. 

Subcase 6.2. Assume f) = 3. We may assume f) ^ Uq+2/3 (otherwise (7a) applies). 
Therefore, f) projects nontrivially to U2/3 + U2a+2/3 • However, because l?]^^ 7^ x^y^, 
for every nonzero 2; G 3 , we know that Vn\X2j3 — and Vr\U2a+2p ~ . Because 2/3 , 
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q;+2/3, and 2a-\-2(3 all differ by multiples of a, we conclude that Na{H) = kei{a) , 
so (7b) applies. 

Case 7. Assume 6.1(7). 

Subcase 7.1. Assume Na{H) = keT{a) . Because a + (3 is the only root that differs 
from /9 by a multiple of a, we must have f) = (l) fl (u^ + Ua+fs)) +3. Thus, there 

is some G f), such that = Xy and = Vv, but the projection of w to 
+ Uq,+2/3 + U2q,+2/3 is zero. This contradicts the fact that Xtt,|y^yp + yu,|x„,p + 

2lm{xwylf];;;) 0. 

Subcase 7.2. Assume Na{H) ^ ker(Q;) . Because 2/9, a + 2P, and 2q; + 2/9 all differ 
by multiples of a, we must have 3 = (3 n U2/3) + (30 Ua+2p) + (3 H U2a+2p) ■ Then, 
because \r]z\'^ ^ x^y^ for every nonzero z e 3, we conclude that 3 C Ua+2/3- 

Let V be the projection of t) to + u„+/3. Because (3 and a + /3 differ 
by a, we know that V — {V D Up) + {V D Ua+p) ■ 

Subsubcase 7.2.1. Assume x,, 7^ 0. Because V = {V D Up) + {V (1 Ua+p), there 
is some w ^ V, such that ^ and i/w = 0- For every such w, because 
Xioll/wP + y«,|iCti,p + 2 lm(a;^y^r/^) 7^ 0, we know that Yu, 7^ 0. Thus, we see that 
NAiH) = ker((a + /3) - 2/3) = ker(a - /3) . 

We know that f) n U/3 = , that 1^ projects trivially to Uq, , and that a is the 
only root that differs from /3 by a multiple of a — /3 , so we conclude that i/h — 
for every h & H . 

We now see that (8a) applies. 

Subsubcase 7.2.2. Assume . This is similar to the preceding subsubcase 

(indeed, they are conjugate under the Weyl reflection corresponding to the root a); 
we see that (8b) applies. 

Case 8. Assume 6.1(8). By considering the projection of f) to Uq + U/?, and noting 
that X yh for every h E H , we see that Na{H) — keic{a — (3) . The only other 
pair of roots that differ by a multiple of o; — /3 is {a + /3, 2(5} . Thus, we see that 
(9) applies. 

Case 9. Assume 6.1(9). By considering the projection of () to u^ + Up, we see 
that Na{H) = ker(a; — P) . Because 0^ 7^ for every u E i)\ U2a+2p, but (3 is the 
only root that differs from a by a multiple of a — /3, we conclude that [) projects 
trivially into every root space except u^. Up, and U2a+2p- Thus (10) applies. 

Case 10. Assume 6.1(10). We may assume f) ^ (otherwise (11a) appUes). 
Thus, there is some root a ^ a, such that the projection of \) to Ua is nontrivial. 
However, because 7^ for every nonzero h E f) , we know that f) fl u^- = . 
Thus, A^a(^) = ker(a - a) . 

Because yh = and yn = for every nonzero /i G f) , we know that cr ^ (3 
and a 7^ 2/3. If(j = a + /5 or a = Q; + 2/3, we obtain (lib). If cr = 2q; + 2/5, we 
obtain (11c). 

Case 11. Assume 6.1(11). Because 0„ 7^ and |/„ 7^ 0, we must have Na{H) — 
ker(Q; — (5) . Then, because a-\- (5 does not differ from ct by a multiple of a — (3, 
we conclude that x„ = . 

Because r/^ 7^ 0, but no root differs from a + 2/3 by a multiple of a — /3, we 
conclude that f)nUa+2/3 7^ 0. Because 3 is one-dimensional, this implies z G Ua+2p, 
so = 0. 
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Since = and = 0, we conclude, from the inequality x^||/„p — 
(f^uVuVz + '^^T^iVz^uVi) 7^ 0, that Yu 7^ 0. This is a contradiction, because 2/3 does 
not differ from a by a multiple of a — (3, and f) fl U2/3 = (because, as shown 
above, 3 C Uo,+2p)- ■ 



7 Subgroups that are not contained in A?^ 

Let if be a closed, connected subgroup of G that is not contained in N . 
In this section, we determine whether is a Cartan-decomposition subgroup or 
not (and, if not, we calculate /x(i7)). 

Lemma 7.1 shows that we may assume H C AN, and then Lemma 7.3 
shows that we may assume H satisfies the technical condition of being compatible 
with A. (Both of these lemmas are well known.) Furthermore, we may assume 
that HON is not a Cartan-decomposition subgroup, and that A H (otherwise, 
it is obvious that if is a Cartan-decomposition subgroup). 

Theorem 7.4 describes ^{H) for every such subgroup that is a semidirect 
product {H r\ A) k {H H N); and Proposition 7.6 describes A*(if) for the other 
subgroups (except that the one-dimensional case appears in Lemma 7.8). 

Lemma 7.1. [14, Lem. 2.9] Let H be a closed, connected subgroup of a con- 
nected, almost simple, linear, real Lie group G. There is a closed, connected 
subgroup H' of G and a compact subgroup C of G , such that CH = CH' , and 
H' is conjugate to a subgroup of AN . 

Definition 7.2. Let us say that a subgroup H of AN is compatible with A 
a H C TUCn{T) , where T = An (HN) , U = H n N , and Cn{T) denotes the 
centralizer of T in N . 

Lemma 7.3. [14, Lem. 2.3] If H is a closed, connected subgroup of AN , then 
H is conjugate, via an element of N , to a subgroup that is compatible with A. 

Theorem 7.4. Assume that G = SU(2,n). Here is a list of every closed, 
connected, nontrivial subgroup H of AN , such that H is of the form H = T K U , 
where T is a one- dimensional subgroup of A , and U is a nontrivial subgroup of N 
that is not a Cartan-decomposition subgroup. 

1. Suppose dimu = 1, u = 3, and we have [r^/il^ = x/jY/j for every h E U . 

(a) If u = U2/3 or u — U2a+2/3; then IJ>{H) is described in [I4, Prop. 3.17 or 
Cor. 3.18]. 

(b) Otherwise, T = ker(Q;), and H is a Cartan-decomposition subgroup. 

2. Suppose u = (u n {Ua+i3 + U2/3)) -\- I, I C U2Q+2/3, thcrc is some v E u, 
such that Yt) 7^ 0, and T = ker(a — /5) . Then fi{H) ^ [\\h\\, \\h\\^^^^ , unless 
dim if = 2, in which case p{h) x for every h E H . 

3. Suppose u = (u n {Ujs + Ua+p)) + i, T = ker(Q;), and there is some nonzero 
A G C, such that we have x„ = Ay„ for every u E U , and we have rj^ = iXyz 
and Xz = |A|^Yz for every z E ^. Then H is a Cartan-decomposition 
subgroup. 
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4. Suppose 0„ = and x„ = for every u we have 3 C U2/3 , and u 7^ 3 . 

(^a^ If u = (u n U/3) + 3, then //(i?) is described in [14, Prop. 3.17 or 
Cor. 3.18]. 

(b) Otherwise: 

i. // u = (u n (u/3 + Ua+2/3)) +3; ^/ien T — ker{a + (3) , and p{h) x h 
for every h E H . 

a. If u= (u n (u/3 + U2a+2/3)) + 3 ; i^/^en T = ker(2Q; + f3) , and fi{H) ^ 
, unless d\mH = 2, in which case p{h) x for 

every h & H . 

Hi. 7/3 = and u C + U2/3, ^/len T = ker(/3), anc? H is a Cartan- 
decomposition subgroup. 

5. Suppose yu = 0, y„ = 0, and \xu\'^+2Re{4>Jh) 7^ for every u e U\U2a+2/3- 

(a) If u = (u n Uq+z?) + 3; ^/len /^(i7) is described in [I4, Prop. 3.17 or 
Cor. 3.18]. 

(b) If u C Ua+f3 + U2a+2f3 , but u ^ {uD Ua+fi) + 3 ; then T — ker (a + /3) , and 
H is a Cartan- decomposition subgroup. 

(c) If u = {u n (u„ + Ua+i3 + u„+2/3)) +3, but u ^ u^+p + U2a+2/3 , then 
T — ker(/3), and p{h) x h for every h & H . 

6. Suppose u = (u n (Ua + U2/3)) + (u n Ua+is) , T — ker(Q; — 2/3) , u ^ Uq+z? , and 

there is some nonzero 00 ^ such that 0„ = </)oy« /or eweri/ u eU . Then 
l^{H) ^ [\\hl /3j ^ jjj^^ _ 2, m which case, p{h) x \\hf/^ for 

every h E H . 

7. Suppose 0u = and dimc(x„, ^ 1 /or ewerT/ u E U, and \r]z\^ 7^ XzVz; 
/or every nonzero z E ^. 

(a) If UG Ua+2/3, then /J^iH) is described in [I4, Prop. 3.17 or Cor. 3.18]. 

(b) If u Ua+2/3, and u = (u fl (u^ + Ua+pi) + 3; then T — ker(Q;), and 
p{h) X for every h E H . 

8. Suppose 0u = for every u E U , there exist nonzero ^1,^2 G u, such 
that dime (x^i, 1/^1) 7^ 1 and dimc(a;^2) I/j^q) = 1? ^'^'^ /lawe x^2ll/t;2p + 
y*;2k^;2l^ + 2 Im(x^2yt^77;;7) 7^ /or even/ such V2 E u, and jjy^p 7^ x^y^, /or 
ewerT/ nonzero z E 

(a) If u = (u n {Ua+i3 + U2/3)) + (u n Ua+2/3), then T = ker(a — /?) and 

Mi^)^ [ll^ll'^MI^III- 

If u = (u n (u/3 + U2a+2/3)) + (u fl Uq+2/3), ^/icn T = ker(2a + /?) and 

5. Suppose dimu < 3, U = (un(UQ+U/3)) + (ur\{Ua+i3 + U2p)) , Un(Ua + U/3) 7^ 0, 
and we have 0„ x y^ and x y^ for u E U . Then T = ker(a — and 
p(/i) X /or every h E H . 
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10. Suppose dimu — 2, i = U2a+2/3, <Pu ^ <^^<^ Uu ^ fof every u & U \ Z . If 
u= {ur){Ua + U/3)) +3, then T = keT{a - P) , and n{H) ^ [\\h\\, \\hf/^] . 

11. Suppose dimu < 2 and (j)u 7^ 0, i/u — 0, — 0, and + 2Re{4>uThi) = 
/or ever?/ nontrivial u & U . 

(a) If UG Ua, then /J^iH) is described in [14, Prop. 3.17 or Cor. 3.18]. 

(h) // u C Uq + Ua+/3 + Uo,+2/3, but u ^ ; then T — ker(/3), and H is a 
Cartan- decomposition subgroup. 

(c) Ifu C Ua + U2a+2/3; but u ^ ; ^/icn T — ker(Q; + 2/3), ano? x 
/or e?;e?i/ h & H . 



Proof. For h G i7, wc wish to approximately calculate ||p(/i)||- We write 
h = au with a E T and u G f/. Writing a = diag(ai,a2, . . . ,a„+2), we always 
assume either that Oi > 1 or that oi = 1 and a2 > 1 (perhaps replacing h 
with — because = this causes no harm). 

Because T normahzes U , we know that C/ is a subgroup that is hsted in 
Corollary 6.2, and we have T C Ng{U). This leads to the various cases hsted in 
the statement of the theorem. 

(lb) We have p{u) x -u for m G t/ and p{a) x ||a||^ for a G T, so is a 
Cartan-decomposition subgroup. 

(2) We have |0«| + ||/„| + |r7„| + |x^,| = and y„ = 0{xu), so Uij = 0[l + \xu\) 
whenever (i, j) ^ (1, n + 2) . Then, because ai — al, we see that 



Ui 



0[a2(l + \Xu\)] = + \h^,r^+2\'/') = 0{\\h\\'/') 

0(||/ip/^). This completes the proof if 



whenever i > 1. Therefore p{h) 
dim// > 2 (that is, if dimC/ > 1). 

If dimC/ = 1, then y„ x x„ and x„ = 0. We have \\h\\ — ai(l + , 



A(/i) = aia2 



^ 1 2 1*'''"! y** 



2\3/2 



and 



det 



2,1 



^2,2 



3/2 

= 0102 = O^ . 



Thus, Wkf/"^ = 0{p{h)). We conclude that p{h) x 

(3) Replacing H hj a conjugate under Ua , wc may replace H with a similar 
subgroup i/' with A = 0. Thus, H' = T « U' with U' C ?7y3^2/3. Then [14, 
Prop. 3.17] implies i/ is a Cartan-decomposition subgroup. 

(4(b)i) The Weyl reflection corresponding to the root a conjugates if to a 
subgroup of type (5c). 

(4(b)ii) The Weyl reflection corresponding to the root a conjugates H to 
a subgroup of type (2). 

(4(b)iii) [14, Prop. 3.17] implies if is a Cartan-decomposition subgroup. 

(5b) [14, Prop. 3.17] implies H is a, Cartan-decomposition subgroup. 
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(5c) We have 

{0(1) if i ^ 1 and ^ + 2 

0{aix) if i = 1 and j n + 2 

0{x) if i 7^ 1 and j = n + 2 

and /ii,n+2 ^ + |x|) . We conclude that p{h) x h. 

(6) From the proof of 4.3(5), we know that u x Ui^n+2, that Uij — 
whenever ^ (l,n + 2), and that Uij — 0(^\\u\\^^^) whenever 

i ^ 1 and j ^ n + 2. (In particular, /i x ai(l + 1*1,71+2) ■) Furthermore, we have 
ai — al- Therefore 

p(h) X aia2p(u) x al^^p{u). 

The desired conclusion follows. 

(7b) From Lemma 5.4, we know = 0(l + |A(w)|) . Then, because 

■1 . f hi^i hi,2\ 2 

and A{au) = o^A(n), we have 

\\hr = 0{al\\ur) = 0{al + \A{h)\) = 0{p{h)). 

(8) Assume (8a). (The other case, (8b), is conjugate to this one by the 
Weyl reflection corresponding to the root a.) From Lemma 5.4, we have = 
0(1 + |A(ii)|) . Then, because Oi = a|, we have 

\\hf/^ = aia2||H||'/' = 0{aia2 + \A{h)\) = 0{p{h)). 

(9) From the proof of 4.3(6), we know p{u) x 1 + A{u) x The 
proof is completed as in (8). 

(10) Because (ffu^Uu, it is easy to see that 

/i X ai(l + \(f>u\'^\yu\'^ + Kl) X ai(l + \(f>X + K\) 

and 

A{h) X aia2(|?/„h0.r + KM X af + = 
Then it is not difficult to see that p{h) = 0(||/i||^/^) for every h e H. So 

p{H)^[\\h\\,\\hr/']. 

(lib) We have p(a) x a for a e T and p{u) x for m e ^7, so if is a 
Cartan-decomposition subgroup. 

(11c) H is conjugate (via an element of Ua+213) to T K Ua- From [14, 
Prop. 3.18], we have p{h) x \\hf for every heT xUa- Therefore p{h) x 
for every h e H . m 
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Lemma 7.5. [14, Lem. 2.4] Assume G — SU(2,n), and let H be a closed, 
connected subgroup of AN that is compatible with A . Then either 

1. H ={HnA)x (HnN); or 

2. there is a positive root to , a nontrivial group homomorphism i/j: kero; 
UujU2uj, and a closed, connected subgroup U of N , such that 

(a) H — { aip{a) | a e ker a; }[/; 

(b) U n ■0(kera;) — e; and 

(c) U is normalized by both kero; and ■0(kera;). 



Proposition 7.6. Assume that G = SU(2,n). Let H be a closed, connected, 
nontrivial subgroup of AN , that is compatible with A, such that 

• H n N is not a Cartan- decomposition subgroup; 

• H ^ {HnA){HnN); and 

• dim H > 1. 

Then there are positive roots cu and a, and a one- dimensional subspace j: of 
(kero;) + u^^ + U2uj, such that ^=j;+(^nn), l^nnCU(j + U2a, and either: 



1. laJ = a, a = a + P , and fi{H) ^ [II^H' H^H^/ II ^11)] / or 

2. u^a a^a + 2p, and n{H) ^ V(log \\h\\f, \\hf] ; or 

3. u; = (3, a^a + 2(3, and ij{H) ^ [\\h\\ (log ||/i||)'^^ \\hf] , where 

2 otherwise 



or 



4. u; = f3, a = a + (3, and fi{H) f« (log , where r is defined as 
above; or 

5. un {Uu, + U2uj) ^ 0, in which case H is a Cartan- decomposition subgroup. 



Proof. We use the notation of Lemma 7.5: T = kera;, U = H f] N , i/j: T ^ 
Uu,U2uj, and H = {aip{a)} x U . 

We need only consider the cases in Corollary 6.2 for which H (now called U) 
is normalized by the kernel of some (reduced) positive root. Here is a list of them. 

1. Na{U) = ker(/3): 6.2(4(b)iii), 6.2(5c), and 6.2(llb). 

2. Na{U) = ker(a + (3): 6.2(4(b)i) and 6.2(5b). 

3. NAiU) = ker(Q;): 6.2(lb), 6.2(3), and 6.2(7b). 

4. Na{U) =ker{a + 2(3): 6.2(llc). 
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5. Na{U) = A: 6.2(la), 6.2(4a), 6.2(5a), 6.2(7a), and 6.2(lla). 

Note that in each of the cases with Na{U) — A, there is a (reduced) positive 
root a, such that u C u^- + U2a- 

Case 1. Assume uj — (5 . 

Subcase 1.1. Assume 6.2(4(b)iii). Prom (7.7), we know that if is a Cartan- 
decomposition subgroup. 

Subcase 1.2. Assume 6.2(5c). There is some u ^ U, such that 0„ ^ 0. Then, 
because ip{T) C U13U213 normahzes t/, we must have U fl Ua+2p 7^ e. This is a 
contradiction. 

Subcase 1.3. Assume 6.2(llb). Let u G u. Because U is normahzcd by ip{T), 
there is some nonzero u G + U2/3, such that v normalizes u; thus, [u,v] G u. 
Then, because 4)[u,v] = 0, but 0/i 7^ for every nontrivial h & U ,we conclude that 
[u,v] — 0. However, 0^ 7^ 0, and either 7^ or 7^ 0, so either X[u,v] 7^ or 
V[u,v] 7^ . This is a contradiction. 

Subcase 1.4- Assume Na{U) — A. There is a positive root cr, such that u C 

If a = (3 , then, from (7.7), we know that if is a Cartan-decomposition 
subgroup. 

Suppose a — a + 2P . Clearly \\h\\ xai|?]„|. Also, 

p{h) X ai|7y„|^ + ai(logai)'', 

where r = 1 if i/^iT) C C/2/3 (i-e., if yn — for every h E H) and r = 2 if 
fp{T) <f_ U213. The smallest value of ||p(/i)|| relative to \\h\\ is obtained by taking 
rju X (logoi)^/^, resulting in p{h) x (log . Then, since p{u) x for 

u E U , we conclude that p{H) ^ [\\h\\ (log ||/^||)"^^ \\hf] . 

Because U is normalized by the nontrivial subgroup ip{T) of U13U213, we 
know that a ^ a. Therefore, we may now assume a = a + p. We show that 
p{H) fti [\\h\\, II (log II /ill)'"] . For u eU , we have p{u) x u. For a G T, we have 

p{aip{a)) X ||a||(log||a||)'' X ||aV'(a)||(log ||aV'(a)||)^ 

All that remains is to show that p{h) = O [\\h\\ (log ||/i||)'^] for every h E H . 
Because p{au) x an for every an G TU (see [14, Cor. 3.18]) and ||'0(q)|| ^ 
||V'(a)~i X (log||/^||)^ we have 

p{h) = p(^(a))p(a«)=0[||p(^(a))||||p(a..)||] 
= 0[(log||a||)n|a«||] =0[(log||/i||)n|/i||]. 

Case 2. Assume u — a -\- (3 . The Weyl reflection corresponding to the root a 
conjugates each of 6.2(4(b)i) and 6.2(5b) to a subgroup with uj — (3. 

Thus, we may now assume Na{U) = A. li a ^ a, then the Weyl reflection 
corresponding to the root a conjugates H to a subgroup with u = f3 . If o" = a, 
then the Weyl reflection corresponding to the root (3 does not change uu, but 
conjugates if to a subgroup Hi with a — a + 2(3 . Then (as we already observed) 
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the Weyl reflection corresponding to the root a conjugates Hi to a subgroup with 

Case 3. Assume uj = a. Because U must be normahzed by the nontrivial subgroup 
'4>{T) of Ua, we see that U cannot be of type 6.2(lb) or 6.2(3). 

Subcase 3.1. Assume 6.2(7b). Because U must be normahzed by the nontrivial 
subgroup ip{T) of Ua, we see that = for every u e [/, so u = 3. Thus, again 
using the fact that U is normahzed by '4>{T), we see that u C Uq+2/3 + U2Q+2/3, 
and the projection of u to Ua+2/3 is one-dimensional. For every ^ e u, we see that 
7^ (because \r]z\'^ 7^ x^y^). Thus, we conclude that dimu = 1. Therefore H is 
conjugate under Ua to a subgroup of type 6.2(7a) (considered in Subsubcase 3.2.2 
below) . 

Subcase 3.2. Assume N^iU) = A. If cr = a, then (7.7) implies that H is a Cartan- 
decomposition subgroup. Because U is normalized by the nontrivial subgroup 
i/^iT) of Ua, we know that a ^ (3 . 

Subsubcase 3.2.1. Assume a = a + (3 . We have 

^oi ai0^(a) aiXu -|ai|x„p + ioiXu^ 
h — a'4>{a)u — \ a-i 



We have x oilogai + ai|xu| + a\\xu\ and, for i > 1, we have hij - 
0{ai + \xu'\) ■ The largest value of ||p(^)|| relative to \\h\\ is obtained by takin 
logai X \xy\'^ (and x„ small), which yields p{h) x a\\ogai x ||/;,p/lo. 
Because p{u) x m for m e t/, we conclude that n{H) log 

Subsubcase 3.2.2. Assume a — a -\- 2(3 . We have 

^ai ai0^(a) air]u -ai(l)^(a)rh' 
h — aip{a)u — \ ai —aifj^ 



We have /i x (l + ai||-?/'(a)||) (l + \i]u\) and p{h) x a^(l + (note that 

det ( Ji,n+2 \ ^ Q-) rpj^g smallest value of \\p{h)\\ relative to \\h\\ is obtained 

by taking rju — 0{1), which results in p{h) x x ||/i|p/(log Because 
p{u) X \\uf for ueU,we conclude that n{H) ^ [\\hf/{\og ||/^||)^ \\hf] . 

Case 4- Assume oj = a + 2/3. The Weyl reflection corresponding to the root /? 
conjugates 6.2(llc) to a subgroup H' with u; — a (of type 6.2(7b) with f)' = 3' c 

lla+2/3 + U2a+2/3 ) ■ 

Thus, we may now assume Na{U) = A. \i a ^ (3, then the Weyl reflection 
corresponding to the root j3 conjugates H to a subgroup with uj = a. Now assume 
a = p. The Weyl reflection corresponding to the root a does not change a;, but 
conjugates H to a subgroup Hi with a — a + 2P . Then (as we already observed) 
the Weyl reflection corresponding to the root P conjugates Hi to a subgroup with 
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Lemma 7.7. Assume G is a connected, almost simple, linear, real Lie group 
of real rank two. Let H be a closed, connected, nontrivial subgroup of AN , such 

that H is compatible with A, and H ^ [H n A){H n N) . We use the notation of 
[14, Lem. 24]: T = kcrcj, H = T kU , ip-.T^ U^U2u^, and H = {a^(a)} x U. 
If uH (U(^ + U2ui) 7^ 0, then H is a Cartan-decomposition subgroup. 

Proof. By passing to a subgroup of H , there is no harm in assuming u fl 
(Uo, + U21S). We use the notation of the proof of [14, Prop. 3.17]. For each 
a e T, clearly fiMA{ai^{a)U) D iiMA{aip{a))A^ , so iima{H) D iiMA{{ai){a)])A'^ . 
Beause jiMA{T) = T is a line perpendicular to A^^, and fiMA{(i"'Pi'^)) is logarithmi- 
cally close to this line, it is clear that fiMA{(i'4'{<^))^w contains all but a bounded 
subset of the region C. Therefore /i(-f^) contains all but a bounded subset of 
so if is a Cartan-decomposition subgroup. ■ 

Lemma 7.8. (cf. [14, Prop. 3.16(3)]) Assume that G = SU(2,n), and let H be 
a nontrivial one-parameter subgroup of AN , such that H is compatible with A, 
but H ^ (H n A){H n N) . 

Then there is a ray R in , a ray R' in A that is perpendicular to R, 
and a positive number k, such that 

fx{H) ^{rs\reR, seR', \\s\\ = (log \\r\\)'' }. 

8 McLximum dimensions of the subgroups 

For convenience of reference. Tables 1, 2 and 3 hst the (approximate) Cartan 
projection of each subgroup of AN that is not a Cartan-decomposition subgroup. 
The maximum possible dimension for a subgroup of each type is also listed. (These 
dimensions are used in applications to the existence of tessellations.) 

Remark 8.1. Here are brief justifications of the dimensions listed in Tables 1, 
2 and 3. 

6.1(1) By assumption, we have dimiJ — 1. 

6.1(2) Let p: — > Ua+p be the natural projection. Then kerp = 3 C U2a+2^, 

so 

dimf) < (dimUa+p) + (dimu2a+2/3) = 2(n — 2) + 1 = 2n — 3. 
6.1(3) We may assume A = 0. Then () C U/3 -|- U2^. So 

dimf) < (dimU/3) + (dimU2/3) ^ 2(n - 2) + 1 ^ 2n - 3. 

It is easy to construct an algebra of this dimension, with or without an element u 
as described in (3a). 

6.1(4) Let V be the projection of 1) to Uq, + Uq,+^ + Uq,+2/3- Because (l)fj is a 
form of signature (2, 2) on Uq -|- Ua+2/3 , we know that dim(y fl (Uq -|- Ua+2/3)) < 2 . 
Thus we have 

dim I) < dim V + dim U2a+2^ < (dim Ua+p + 2) + dim U2a+2/3 
= (2(n - 2) + 2) + 1 = 2n - 1. 
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reference Cartan projection 

6.1(1) p{h) X h 

6.1(2) pi{H)^[\\hl\\hr/'] 

6.1(2)* p{h) X l|/?,l|3/2 

6.1(3a) p{H)^[\\h\\,\\hf/^] 

6.1(3a)* p{h) X ||/if/2 

6.1(3b) p(/i) X h 

6.1(4) p(/i) X h 

6.1(5) M^) ^ oi^iui^r^'] 

6.1(5)* p(/i) X 



6.1(6) p(/i) > 

6.1(7) p{H) ^ [\\hr/M\hr] 



6.1(8) p(/i) 



3/2 



6.1(9) p(H)^[\\h\\,\\hf/'] 
6.1(10) p(/^) X \\hr 

6.1(11) Mi/) « [||/.||VM|/,||2] 

Table 1: The subgroups of N that are not Cartan-decomposition subgroups. 




6.1(5) Consider p: {) — > Uq. Because 3 = 0, we have dimkerp < dimUa+^ = 
2n — 4. Because p(f)) C ]R0o, we have dimp(l^) < 1. Thus, diml^ < 2n — 3. 
6.1(6) See Lemma 8.2 below. 
6.1(7) See Lemma 8.3 below. 
6.1(8) See Lemma 8.4 below. 

6.1(9), 6.1(10), 6.1(11) are obvious from the statements. 

7.4(la) Because dimu = 1, we have dim() = dimt + dimu = 2. 

7.4(2) The kernel of the projection from u to Ua+p is 3, so dimf) = 
1 + dimC/ < 1 + (1 + dimUa+13) = 2n - 2. 

7.4(4) dim I) = 1 + dimu < 1 + (dimu^ + dim^) = 2n - 2. 

7.4(5a) dimJ) < dim t + dim u„+/3 + dim3 < 1 + (2n - 4) + 1 = 2n - 2. 

7.4(5c) Add 1 (the dimension of T) to the bound in 6.1(4). 

7.4(6) Add 1 (the dimension of T) to the bound in 6.1(5). 

7.4(7a) dim {) < dim t + dim Ua+2f3 = 1 + 2 = 3. 

7.4(7b) Add 1 (the dimension of T) to the bound in 6.1(6). 

7.4(8a) Because 7^ for every nonzero u e uD {Ua+p + U2/3), we have 
dim(u n {Ua+f3 + U2/3)) < 1 . Therefore dim f) < dim t + 1 + dim Ua+2(3 — 4 . 

7.4(8b) This is conjugate to 7.4(8a), via the Weyl reflection corresponding 
to the root a. 

7.4(9) Add 1 (the dimension of T) to the bound in 6.1(8). (To achieve this 
bound for n > 4, choose u,u E uH (Uq + Up) in the proof of Lemma 8.4. 
7.4(10) and 7.4(11) are obvious from the statements. 
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reference 


Cartan projection 


maximum dimension 


7.4(la) 


M^)~ [II^IUI^III 


2 


7.4(2) 


/ -r -r\ ril» II ll» IIQ/OI 


2n - 2 


7.4(2)* 


/ 1 \ 1 1 1 1 1 o /o 

p{h) X ||/lf /2 


2 


7.4(4a) 


/.(//) ^ [II^IUI^III 


2n-2 


7.4(4(b)i) 


p{h) X /i 


2n-2 


7.4(4(b)ii) 


Fiif II lit iiQ/0~1 

p{H)^ [\\h\\,\\hf/^] 


2n - 2 


7.4(4(b)ii)* 


/ 1 \ II 7 1^/9 

p{h) X \\hf/^ 


2 


7.4(5a) 


piH)^[\\h\\,\\hr] 


2n-2 


7.4(5cj 


p[n) X 


zn 




A*v-" J ~ Lll'^ll) ll'^ll J 


9n 

z/i — z 


7 4(6)* 


p(h) X 

rV"'/ ll"^ll 


2 


7.4(7a) 


Mi^) « OI^IIMI^II'] 


3 


7.4(7b) 


\\hr 


1 '^'H 71 PA/'PTI 

1 2r7 — 2 r? odd 




,.(n\ ~ ril7)l|3/2 ||7,||21 

p{n} ~ , ||/7,|| J 


A 




n(U\ ~ ril7)l|3/2 ||/,||2] 


A 


7.4(9) 


p{h) X ||/i||3/2 


{a n > 4 
^3 n = 3 


7.4(10) 




3 


7.4(lla) 




3 


7.4(llc) 


Pih) X 


3 



Table 2: The subgroups of AA^ that are not Cartan-decomposition subgroups, and 
are a nontrivial semidirect product T \kU . 



reference 




7.6(1) 




7.6(2) 




7.6(3) (r = 


1) 


7.6(3) (r = 


2) 


7.6(4) (r = 


1) 


7.6(4) (r = 


2) 


7.8 





Cartan projection maximum dimension 

M//)^ [|I^IUI^IlV(log||/^||)] 2n-2 

Mi/)^ [II^IIV(iog||/^ll)M|/^|p] 2 
Mi/)^ [ll^ll(i°g|l^ll)'^''ll^ll'] 3 

p{H)^ [\\h\\i\og\\h\\),\\hf] 3 

Mi^)^ Ol^lUI^II(logll^ll)] 2n-2 

Mi^)^ Ol^lUl^llOogll^ll)'] 2n-3 
p{h)^\\hf{\og\\h\\)^^ 1 



Table 3: The subgroups of AN that are not Cartan-decomposition subgroups and 
are not a semidirect product of a torus and a unipotent subgroup. 
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7.6(1) dim f) < 1 + dim(Ua+/3 + U2a+20) = 2n - 2 . 

7.6(2) Because ■0(^) normalizes (hence centralizes) U, the subgroup U 
cannot be all of Ua+2p , so dim U < 1. Therefore dim H — 1 + dim U <2. 
7.6(3) dim^ < 1 + dimUa+2/3 = 3. 

7.6(4) Because il^{T) normalizes (hence centralizes) U , the projection of u 
to Ua+/3 cannot be all of Ua+/3 if ^ U213, that is, if r = 2. Therefore dimL'" < 
dim{Ua+i3+U2a+2(3)~{r — l) — 2n — 2 — r. Therefore diml^ = 1+dimC/ < 2n — l—r. 

Lemma 8.2. The maximum dimension of a subalgebra of type 6.1(6) is as 
stated in Table 1. 



Proof. We begin by showing that dimf) < 2n — 1 (cf. [16, Lem. 5.8]). Let V 
be the projection of i) io Up + Ua+p ■ Because dim 3 < 3 , we just need to show 
that dimK <2n — 4. Because V does not intersect (or Uq+^, either, for that 
matter), and U/j has codimension 2n — 4 in U/3 + Ua+13, this is immediate. 

When n is even, there is a subgroup of dimension 2n — 1 . (For example, 
the N subgroup of Sp(l, n/2) . More general examples are constructed in [15, §4].) 

Let us show that if n is odd, then dim H < 2n—3. (Our proof is topological; 
we do not know an algebraic proof.) Suppose that dim if > 2n — 2 (this will lead 
to a contradiction). Because dim 3 < 3, we have dim [3/3 > 2n — 5. Thus, there is 
a {2n — 5) -dimensional real subspace X of C"~^ and a real linear transformation 
T: X — > C"~^, such that x and Tx are linearly independent over C, for every 
nonzero xeX (cf. [16, Cor. 5.9]). Thus, if we define U : X ^ C"-^ by Ux^ix; 
then X, Tx, and Ux are linearly independent over M, for every nonzero x & X . 
Thus (writing n = 2/c + 3): there is a {Ak + 1) -dimensional real subspace X 
of M'^*^"'"^ and real linear transformations T,U : X ^ 1^^^^+^ , such that x , Tx , and 
Ux are linearly independent over M, for every nonzero x & X . There is no harm 
in assuming X — ]R^*^+^ (under its natural embedding in ]R^*^+^ ) . 

Let E = (5^^ X K'''''+^) / ~ , where {x, v) ~ {—x, —v) , and define a continuous 
map (: E ^ MP^^' by (^{x, v) = [x] , so {E, () is a vector bundle over MP^'^. Then 
{E,Q = r © © 7]^, where r is the tangent bundle of M.P^^ , is a trivial 
line bundle, and 7]^ is the canonical bundle of MP""^. (To see this, note that the 
subbundle 

{{x,v) e S'^'' xR^''+^ \v ±x}/r^ 
is the total space of r [13, pf. of Lem. 4.4, pp. 43-44], the subbundle 

{ {x, v) e S^'' X M^*^+i I V e Ma; }/~ 

has the obvious section x {x, x) , and the subbundle {S^'' x (0 x M)) / ~ IS 
isomorphic to 7]^ via the bundle map (x, (0,t)) t-^ {x,tx).) Therefore, letting a 
be a generator of the cohomology ring H*(RP^''; Z2) , we see that the total Stiefel- 
Whitney class of {E,C) is w ^ (1 + a)^'=+Hl)(l + a) = (l + a)^*^+2 [13, Eg. 2, p. 43, 
and Thm. 4.5, p. 45], so 

^(4fc+2)-3+i = w,k = (^^^^ ^) a'" = {2k + l){4k + l)a^^ ^ 

(because {2k + 1)(4A; -|- 1) is odd). Therefore, there do not exist three pointwise 
hnearly independent sections of {EX) [13, Prop. 4, p. 39]. 
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Any linear transformation Q : M^'^+-'^ — > ]R^*^+^ induces a continuous function 
Q: S"^^ ^ ^^^=+2, such that Q{-x) = -Q{x) for all x e S"^^- that is, a section of 
[EX)- Thus, Id, T, and U each define a section of (EX)- Furthermore, these 
three sections are pointwise linearly independent, because x, Tx, and Ux are 
linearly independent over M, for every x G S"^^ . This contradicts the conclusion 
of the preceding paragraph. ■ 

Lemma 8.3. The maximum dimension of a subalgebra of type 6.1(7) is as 
stated in Table 1. 

Proof. Replacing iJ by a conjugate under U-a) , we may assume Xy = 0. 
Therefore = for every ,2 G 3. (Thus, in particular, we have dim 3 < 2.) 

For the projection p: f} — > Ua+p, we have kerp = Wv + 3. (There cannot 
exist a linearly independent v'; otherwise, replacing v' by some linear combination 
with V, we could assume x„/ = 0, which is impossible.) Thus, dimkcrp < 3. 

Because x^ = for every 2; G 3, p([)) must be a totally isotropic subspace for 
the symplectic form lm{xx^) , so dimp(f)) < n — 2. Therefore dim f) < (n — 2) + 3 = 
n + 1. 

For n > 4 , here is an example that achieves this bound: 

X, Xi, . . . ,Xn-2 e K, I 
77GC j 

For i> G f), we claim that dimc{xv,yv) — 1 only if either = or = 0. (In 
either case, it is clear from the definition of [) that cither x„|yup 7^ or y„|a;„p 7^ 0, 
respectively.) Suppose dimc{xv,yv) = 1, with x.^ 7^ and y^ ^ 0. There is some 
nonzero A G C, such that = Xxy. We must have Xi 0. (Otherwise, let 
i G {1, 2, . . . , n — 2} be minimal with Xi ^ 0. Then — yi — Xxi ^ 0, 
contradicting the minimality of i.) Because yi — ix is pure imaginary, but Xi is 
real, we see that A is pure imaginary. On the other hand, ?/2 = is real (and 
nonzero), and X2 is also real, so A is real. Because A 7^ 0, this is a contradiction. 

Now let n = 3, and suppose dimf) = 4. (This will lead to a contradiction.) 
Because equality is attained in the proof above, we must have dimp(l)) = n — 2 — 1 
and dim 3 — 2. In particular, there exists w e i) with x^ 0. For i G IR, let 
Wt — w + tv . Then 

^tlVwtl^ + Vwtlxwtl'^ + '^'i-M^wtyltVwt) = tWyy\^ + o{t^) 

Thus, this expression changes sign, so it must vanish for some t. This is a 
contradiction, because dimc(xtt,j, y^^) = 1 for every t. ■ 

Lemma 8.4. The maximum dimension of a subalgebra of type 6.1(8) is as 
stated in Table 1. 



a;2 
f) = <( I ix xi 



■ Xn-2 V «X 
• Xn-3 iXn-2 -Tj 




+x^,oo as t ^ 00 
— x„oo as t — > —00. 
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Proof. For n > 4 , here is the construction of 3-dimensional subalgebras of n of 
this type. Let (f) — 1 and ^ — i. Choose y,y,x,x e C""^, r],fj e C, and x,x e R, 
such that 

= = 3iyyt ^ 0. (8.1) 

Now, choose y,y G M, such that 

lm{yx^ — iyx'^ + yx^ — yx^ + iy) = (8.2) 

and 

Iin(yx^ — iyx^ + iyx^ — iyx^ + iy) = 0. (8.3) 

Define u,u as in Eq. (2.3), and let v = [u,u]. Then y^ 7^ and Xy 7^ 0, but, from 
Eq. (8.1), Eq. (8.2) and Eq. (8.3), we have [v,u] = [v,u] — 0. Thus, we may let f) 
be the subalgebra generated by u and u. (So {u, u, v} is a basis of 1^ over R.) 

Note that, because \yy'^\ = |l/P/3 7^ we know that y and y must be 
linearly independent over C. Thus, these 3-dimensional examples do not exist 
when n = 3. ■ 
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